Encoding sequence encoding method and device thereof, and decoding method and device thereof

ABSTRACT

Provided is a video decoding method including obtaining encoding order information indicating whether an encoding order of a first block and a second block that are adjacent to each other is changed; determining the encoding order of the first block and the second block, based on the encoding order information; and decoding the first block and the second block, according to the determined encoding order.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a Continuation of U.S. patent application Ser. No. 15/771,267,filed May 3, 2018, which is a National Stage of InternationalApplication No. PCT/KR2016/013527, filed on Nov. 23, 2016, claimingpriority based on Provisional Application No. 62/259,374 filed Nov. 24,2015, the disclosures of which are incorporated by reference herein intheir entirety.

TECHNICAL FIELD

The present disclosure relates to methods of encoding and decoding avideo, and more particularly, to an intra or inter prediction techniquefor methods and apparatuses for determining encoding and decoding ordersregarding an image.

BACKGROUND ART

When a video of high quality is encoded, a large amount of data isrequired. However, since a bandwidth allowed for transmission of thevideo data is limited, a data rate applied to transmission of the videodata may be limited. Therefore, for efficient transmission of videodata, there is a need for video data encoding and decoding methods withminimal deterioration in image quality and increased compression rates.

Video data may be compressed by removing spatial redundancy and temporalredundancy between pixels. Since adjacent pixels generally have a commoncharacteristic, encoding information of a data unit consisting of pixelsis transmitted to remove redundancy between the adjacent pixels.

Pixel values of the pixels included in the data unit are not directlytransmitted but information regarding a method of obtaining the pixelvalues is transmitted. A prediction method of predicting a pixel valuethat is similar to an original value is determined for each data unit,and encoding information regarding the prediction method is transmittedfrom an encoder to a decoder. Since a prediction value is not completelyequal to the original value, residual data of a difference between theoriginal value and the prediction value is transmitted from the encoderto the decoder.

When prediction is exact, a data amount of the encoding information forspecifying the prediction method is increased but a size of the residualdata is decreased. Therefore, the prediction method is determined bytaking into account sizes of the encoding information and the residualdata. In particular, a data unit that is split from a picture hasvarious sizes, and in this regard, when a size of the data unit isincreased, there is an increased probability that accuracy of predictionis decreased, whereas a data amount of encoding information isdecreased. Thus, a size of a block is determined according tocharacteristics of a picture.

The prediction method includes intra prediction and inter prediction.The intra prediction involves predicting pixels of a block from adjacentpixels of the block. The inter prediction involves predicting pixels byreferring to pixels of a different picture referred to by a pictureincluding the block. Therefore, spatial redundancy is removed throughthe intra prediction, and temporal redundancy is removed through theinter prediction.

When the number of prediction methods is increased, an amount ofencoding information for indicating the prediction method is increased.Thus, when the encoding information to be applied to a block ispredicted from a different block, the amount of the encoding informationmay be decreased.

Since loss of video data is allowed to the extent that the human eyecannot recognize the loss, residual data may be lossy-compressedaccording to transformation and quantization processes, and by doing so,an amount of the residual data may be decreased.

DESCRIPTION OF EMBODIMENTS Technical Problem

Provided is a video encoding method of encoding a video by determiningan optimal encoding order of blocks. Provided is a video decoding methodof decoding the video according to the determined encoding order.Provided is a computer-readable recording medium having recorded thereona program for executing the video encoding method and the video decodingmethod, by using a computer.

Solution to Problem

Provided is a video decoding method including obtaining encoding orderinformation indicating whether an encoding order of a first block and asecond block that are adjacent to each other is changed; determining theencoding order of the first block and the second block, based on theencoding order information; and decoding the first block and the secondblock, according to the determined encoding order.

Provided is a video encoding method including determining whether anencoding order of a first block and a second block that are adjacent toeach other is changed; encoding the first block and the second blockaccording to whether the encoding order is changed; and outputting abitstream including encoding order information indicating whether theencoding order is changed and encoding information of the first blockand the second block.

Provided is a video decoding apparatus including an encoding orderinformation obtainer configured to obtain encoding order informationindicating whether an encoding order of a first block and a second blockthat are adjacent to each other is changed; an encoding order determinerconfigured to determine the encoding order of the first block and thesecond block, based on the encoding order information; and a decoderconfigured to decode the first block and the second block, according tothe determined encoding order.

Provided is a video encoding apparatus including an encoding orderdeterminer configured to determine whether an encoding order of a firstblock and a second block that are adjacent to each other is changed; anencoder configured to encode the first block and the second blockaccording to whether the encoding order is changed; and an output unitconfigured to output a bitstream including encoding order informationindicating whether the encoding order is changed and encodinginformation of the first block and the second block.

The technical problems of the present disclosure are not limited to theaforementioned technical features, and other unstated technical problemsmay be inferred from embodiments below.

Advantageous Effects of Disclosure

Prediction efficiency with respect to a block may be improved byadjusting an encoding order of the block. Therefore, efficiency ofencoding and decoding a video may be increased.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A illustrates a block diagram of an image encoding apparatus basedon coding units according to a tree structure, according to anembodiment of the present disclosure.

FIG. 1B illustrates a block diagram of an image decoding apparatus basedon coding units according to a tree structure, according to anembodiment.

FIG. 2 illustrates a process of determining at least one coding unitwhen a current coding unit is split, according to an embodiment.

FIG. 3 illustrates a process of determining at least one coding unitwhen a coding unit having a non-square shape is split, according to anembodiment.

FIG. 4 illustrates a process of splitting a coding unit based on atleast one of block shape information and split shape information,according to an embodiment.

FIG. 5 illustrates a method of determining a predetermined coding unitfrom among an odd number of coding units, according to an embodiment.

FIG. 6 illustrates an order of processing a plurality of coding unitswhen the plurality of coding units are determined when a current codingunit is split, according to an embodiment.

FIG. 7 illustrates a process of determining that a current coding unitis split into an odd number of coding units when coding units are unableto be processed in a predetermined order, according to an embodiment.

FIG. 8 illustrates a process of determining at least one coding unitwhen a first coding unit is split, according to an embodiment.

FIG. 9 illustrates that, when a second coding unit having a non-squareshape, which is determined when a first coding unit is split, satisfiesa predetermined condition, a shape of the second coding unit that issplittable is limited, according to an embodiment.

FIG. 10 illustrates a process of splitting a coding unit having a squareshape when split shape information does not indicate splitting of thecoding unit into four coding units having square shapes, according to anembodiment.

FIG. 11 illustrates that a processing order between a plurality ofcoding units may be changed according to a split process of a codingunit, according to an embodiment.

FIG. 12 illustrates a process of determining a depth of a coding unitwhen a shape and size of the coding unit changes, in a case where aplurality of coding units are determined when the coding unit isrecursively split, according to an embodiment.

FIG. 13 illustrates a depth determinable according to shapes and sizesof coding units, and a part index (PID) for distinguishing between thecoding units, according to an embodiment.

FIG. 14 illustrates that a plurality of coding units are determinedaccording to a plurality of predetermined data units included in apicture, according to an embodiment.

FIG. 15 illustrates a processing block that is a criterion indetermining a determining order of a reference coding unit included in apicture, according to an embodiment.

FIG. 16 illustrates a video decoding apparatus involving determining anencoding order of blocks, according to an embodiment.

FIG. 17 illustrates a video encoding apparatus involving determining anencoding order of blocks, according to an embodiment.

FIG. 18 is a diagram for describing the necessity of changing anencoding order of blocks.

FIGS. 19A and 19B illustrate embodiments of a method of determining anencoding order of blocks.

FIG. 20 illustrates a method of comparing coding efficiencies so as todetermine whether to change an encoding order of blocks.

FIG. 21 illustrates a reference sample to be used when a block ispredicted according to an intra mode.

FIGS. 22A to 22E illustrate an intra prediction method to be performedon a current block when a right block of the current block isreconstructed by changing an encoding order of blocks.

FIG. 23 illustrates reference blocks to be used when a block ispredicted according to an inter mode.

FIG. 24 illustrates a video decoding method performed by the videodecoding apparatus, according to an embodiment.

FIG. 25 illustrates a video encoding method performed by the videoencoding apparatus, according to an embodiment.

BEST MODE

Provided is a video decoding method including obtaining encoding orderinformation indicating whether an encoding order of a first block and asecond block that are adjacent to each other is changed; determining theencoding order of the first block and the second block, based on theencoding order information; and decoding the first block and the secondblock, according to the determined encoding order.

MODE OF DISCLOSURE

Advantages and features of one or more embodiments of the presentdisclosure and methods of accomplishing the same may be understood morereadily by reference to the following detailed description of theembodiments and the accompanying drawings. In this regard, the presentdisclosure may, however, be embodied in many different forms and shouldnot be construed as being limited to the embodiments set forth herein.Rather, these embodiments are provided so that this disclosure will bethorough and complete and will fully convey the concept of the presentembodiments to one of ordinary skill in the art.

Hereinafter, the terms used in the specification will be brieflydefined, and the embodiments will be described in detail.

All terms including descriptive or technical terms which are used hereinshould be construed as having meanings that are obvious to one ofordinary skill in the art. However, the terms may have differentmeanings according to the intention of one of ordinary skill in the art,precedent cases, or the appearance of new technologies. Also, some termsmay be arbitrarily selected by the applicant, and in this case, themeaning of the selected terms will be described in detail in thedetailed description of the disclosure. Thus, the terms used herein haveto be defined based on the meaning of the terms together with thedescription throughout the specification.

An expression used in the singular encompasses the expression of theplural, unless it has a clearly different meaning in the context.

When a part “includes” or “comprises” an element, unless there is aparticular description contrary thereto, the part can further includeother elements, not excluding the other elements. Also, the term “unit”in the embodiments of the present disclosure means a software componentor hardware component such as a field-programmable gate array (FPGA) oran application-specific integrated circuit (ASIC), and performs specificfunctions. However, the term “unit” is not limited to software orhardware. The “unit” may be formed so as to be in an addressable storagemedium, or may be formed so as to operate one or more processors. Thus,for example, the term “unit” may refer to components such as softwarecomponents, object-oriented software components, class components, andtask components, and may include processes, functions, attributes,procedures, subroutines, segments of program code, drivers, firmware,micro codes, circuits, data, a database, data structures, tables,arrays, variables, or the like. A function provided by the componentsand “units” may be associated with the smaller number of components and“units”, or may be divided into additional components and “units”

The term “current block” refers to one of a coding unit, a predictionunit, and a transform unit which are currently to be encoded or decoded.In addition, the term “lower block” refers to a data unit split from the“current block”. The term “upper block” refers to a data unit includingthe “current block”.

Hereinafter, a “sample” is data allocated to a sampling location of animage and may mean data that is a processing target. For example, pixelvalues in an image of a spatial domain or transform coefficients on atransformation domain may be samples. A unit including at least onesample may be defined as a block.

The present disclosure will now be described more fully with referenceto the accompanying drawings for one of ordinary skill in the art to beable to perform the present disclosure without any difficulty. In thefollowing description, well-known functions or constructions are notdescribed in detail so as not to obscure the embodiments withunnecessary detail.

FIG. 1A illustrates a block diagram of an image encoding apparatus 100based on coding units according to a tree structure, according to anembodiment of the present disclosure.

The image encoding apparatus 100 includes a largest coding unitdeterminer 110, a coding unit determiner 120, and an output unit 130.

The largest coding unit determiner 110 splits a picture or a sliceincluded in the picture into a plurality of largest coding units,according to a size of a largest coding unit. The largest coding unitmay be a data unit having a size of 32×32, 64×64, 128×128, 256×256,etc., wherein a shape of the data unit is a square having a width andlength in squares of 2. The largest coding unit determiner 110 mayprovide largest coding unit size information indicating the size of thelargest coding unit to the output unit 130. The output unit 130 mayinclude the largest coding unit size information in a bitstream.

The coding unit determiner 120 determines coding units by splitting thelargest coding unit. A coding unit may be determined by its largest sizeand depth. A depth may be defined as the number of times that the codingunit is spatially split from the largest coding unit. When the depth isincreased by 1, the coding unit is split into at least two coding units.Therefore, when the depth is increased, sizes of coding units accordingto depths are each decreased. Whether to split a coding unit isdetermined according to whether splitting the coding unit is efficientaccording to rate-distortion optimization. Then, split informationindicating whether the coding unit has been split may be generated. Thesplit information may be expressed as a form of a flag.

The coding unit may be split by using various methods. For example, asquare coding unit may be split into four square coding units of whichwidth and height are half of those of the square coding unit. The squarecoding unit may be split into two rectangular coding units of whichwidth is half. The square coding unit may be split into two rectangularcoding units of which height is half. The square coding unit may besplit into three coding units in a manner that a width or height thereofis split by 1:2:1.

A rectangular coding unit of which width is twice as large as a heightmay be split into two square coding units. The rectangular coding unitof which width is twice as large as the height may be split into tworectangular coding units of which width is four times larger than aheight. The rectangular coding unit of which width is twice as large asthe height may be split into two rectangular coding units and one squarecoding unit in a manner that the width is split by 1:2:1.

Equally, a rectangular coding unit of which height is twice as large asa width may be split into two square coding units. The rectangularcoding unit of which height is twice as large as the width may be splitinto two rectangular coding units of which height is four times largerthan a width. Equally, the rectangular coding unit of which height istwice as large as the width may be split into two rectangular codingunits and one square coding unit in a manner that the height is split by1:2:1.

When the image encoding apparatus 100 is capable of using two or moresplit methods, information regarding a split method that is usable to acoding unit, the split method being from among the split methods thatare available to the image encoding apparatus 100, may be determined foreach picture. Therefore, only specific split methods may be used foreach picture. When the image encoding apparatus 100 uses only one splitmethod, the information regarding a split method that is usable to acoding unit is not separately determined.

When split information of a coding unit indicates that the coding unitis split, split shape information indicating a split method with respectto the coding unit may be generated. When only one split method isusable in a picture including the coding unit, the split shapeinformation may not be generated. When the split method is determined tobe adaptive to encoding information adjacent to the coding unit, thesplit shape information may not be generated.

The largest coding unit may be split to smallest coding units accordingto smallest coding unit size information. A depth of the largest codingunit may be defined to be an uppermost depth, and a depth of thesmallest coding units may be defined to be a lowermost depth. Therefore,a coding unit having an upper depth may include a plurality of codingunits having a lower depth.

According to a largest size of a coding unit as described above, imagedata of a current picture is split into a largest coding unit. Thelargest coding unit may include coding units that are split according todepths. Since the largest coding unit is split according to the depths,image data of a spatial domain included in the largest coding unit maybe hierarchically split according to the depths.

A maximum depth that limits the maximum number of hierarchicallysplitting the largest coding unit or a minimum size of a coding unit maybe preset.

The coding unit determiner 120 compares coding efficiency ofhierarchically splitting a coding unit with coding efficiency of notsplitting the coding unit. Then, the coding unit determiner 120determines whether to split the coding unit according to a result of thecomparison. When the coding unit determiner 120 determines thatsplitting the coding unit is more efficient, the coding unit determiner120 hierarchically splits the coding unit. However, according to theresult of the comparison, when the coding unit determiner 120 determinesthat not splitting the coding unit is more efficient, the coding unitdeterminer 120 does not split the coding unit. Whether to split thecoding unit may be independently determined from whether a neighboringdifferent coding unit is split.

According to an embodiment, whether to split the coding unit may bedetermined from a coding unit having a large depth, during an encodingprocedure. For example, coding efficiency of a coding unit having amaximum depth is compared with coding efficiency of a coding unit havinga depth that is less than the maximum depth by 1, and it is determinedwhich one of coding units having the maximum depth and coding unitshaving the depth that is less than the maximum depth by 1 is efficientlyencoded in each area of a largest coding unit. According to a result ofthe determination, whether to split the coding units having the depththat is less than the maximum depth by 1 is determined in each area ofthe largest coding unit. Afterward, it is determined which one of codingunits having a depth that is less than the maximum depth by 2 and one ofthe coding units having the maximum depth and the coding units havingthe depth that is less than the maximum depth by 1, the one having beenselected according to the result of the determination, are furtherefficiently encoded in each area of the largest coding unit. The samedetermination process is performed on each of coding units having asmaller depth, and finally, whether to split the largest coding unit isdetermined according to which one of the largest coding unit and ahierarchical structure generated by hierarchically splitting the largestcoding unit is further efficiently encoded.

Whether to split the coding unit may be determined from a coding unithaving a small depth, during the encoding procedure. For example, codingefficiency of the largest coding unit is compared with coding efficiencyof a coding unit of which depth is greater than the largest coding unitby 1, and it is determined which one of the largest coding unit andcoding units of which depth is greater than the largest coding unit by 1is efficiently encoded. When the coding efficiency of the largest codingunit is better, the largest coding unit is not split. When codingefficiency of the coding units of which depth is greater than thelargest coding unit by 1 is better, the largest coding unit is split,and the comparison process is equally applied to split coding units.

When coding efficiency is examined from a coding unit having a largedepth, calculation is large but a tree structure having high codingefficiency is obtained. On the contrary, when the coding efficiency isexamined from a coding unit having a small depth, calculation is smallbut a tree structure having low coding efficiency is obtained.Therefore, in consideration of coding efficiency and calculation, analgorithm for obtaining a hierarchical tree structure of a largestcoding unit may be designed by using various methods.

To determine efficiency of a coding unit according to each depth, thecoding unit determiner 120 determines prediction and transformationmethods that are most efficient to the coding unit. To determine themost efficient prediction and transformation methods, the coding unitmay be split into predetermined data units. A data unit may have one ofvarious shapes according to a method of splitting the coding unit. Themethod of splitting the coding unit which is performed to determine thedata unit may be defined as a partition mode. For example, when a codingunit of 2N×2N (where N is a positive integer) is no longer split, a sizeof a prediction unit included in the coding unit is 2N×2N. When thecoding unit of 2N×2N is split, the size of the prediction unit includedin the coding unit may be 2N×N, N×2N, or N×N, according to the partitionmode. The partition mode according to the present embodiment maygenerate symmetrical data units obtained by symmetrically splitting aheight or width of the coding unit, data units obtained byasymmetrically splitting the height or width of the coding unit, such as1:n or n:1, data units obtained by diagonally splitting the coding unit,data units obtained by geometrically splitting the coding unit,partitions having arbitrary shapes, or the like.

The coding unit may be predicted and transformed based on a data unitincluded in the coding unit. However, according to the presentembodiment, a data unit for prediction and a data unit fortransformation may be separately determined. The data unit forprediction may be defined as a prediction unit, and the data unit fortransformation may be defined as a transform unit. A partition modeapplied to the prediction unit and a partition mode applied to thetransform unit may be different from each other, and prediction of theprediction unit and transformation of the transform unit may beperformed in a parallel and independent manner in the coding unit.

To determine an efficient prediction method, the coding unit may besplit into at least one prediction unit. Equally, to determine anefficient transformation method, the coding unit may be split into atleast one transform unit. The split into the prediction unit and thesplit into the transform unit may be independently performed from eachother. However, when a reconstructed sample in the coding unit is usedin intra prediction, a dependent relation is formed between predictionunits or transform units included in the coding unit, so that the splitinto the prediction unit and the transform unit may affect each other.

The prediction unit included in the coding unit may be predicted throughintra prediction or inter prediction. The intra prediction involvespredicting prediction-unit samples by using reference samples adjacentto the prediction unit. The inter prediction involves predictingprediction-unit samples by obtaining reference samples from a referencepicture that is referred to by a current picture.

For the intra prediction, the coding unit determiner 120 may apply aplurality of intra prediction methods to the prediction unit, therebyselecting the most efficient intra prediction method. The intraprediction method includes directional modes such as a DC mode, a planarmode, a vertical mode, a horizontal mode, or the like.

When a reconstructed sample adjacent to a coding unit is used as areference sample, the intra prediction may be performed on eachprediction unit. However, when a reconstructed sample in the coding unitis used as a reference sample, reconstruction with respect to thereference sample in the coding unit has to precede prediction withrespect to the reference sample in the coding unit, so that a predictionorder of a prediction unit may depend on a transformation order of atransform unit. Therefore, when the reconstructed sample in the codingunit is used as the reference sample, only an intra prediction methodfor transform units corresponding to the prediction unit, and actualintra prediction may be performed on each transform unit.

The coding unit determiner 120 may determine an optimal motion vectorand reference picture, thereby selecting the most efficient interprediction method. For inter prediction, the coding unit determiner 120may determine a plurality of motion vector candidates from a coding unitthat is spatially and temporally adjacent to a current coding unit, andmay determine, from among them, the most efficient motion vector to be amotion vector. Equally, the coding unit determiner 120 may determine aplurality of reference picture candidates from the coding unit that isspatially and temporally adjacent to the current coding unit, and maydetermine the most efficient reference picture from among them. Inanother embodiment, the reference picture may be determined fromreference picture lists that are predetermined with respect to a currentpicture. In another embodiment, for accuracy of prediction, the mostefficient motion vector from among the plurality of motion vectorcandidates may be determined to be a prediction motion vector, and amotion vector may be determined by compensating for the predictionmotion vector. The inter prediction may be performed in parallel on eachprediction unit in the coding unit.

The coding unit determiner 120 may reconstruct the coding unit byobtaining only information indicating the motion vector and thereference picture, according to a skip mode. According to the skip mode,all encoding information including a residual signal is skipped, exceptfor the information indicating the motion vector and the referencepicture. Since the residual signal is skipped, the skip mode may be usedwhen accuracy of prediction is very high.

A partition mode to be used may be limited according to the predictionmethod for the prediction unit. For example, only partition modes for aprediction unit having a size of 2N×2N or N×N may be applied to intraprediction, whereas partition modes for a prediction unit having a sizeof 2N×2N, 2N×N, N×2N, or N×N may be applied to inter prediction. Inaddition, only a partition mode for a prediction unit having a size of2N×2N may be applied to a skip mode of the inter prediction. The imageencoding apparatus 100 may change a partition mode for each predictionmethod, according to coding efficiency.

The image encoding apparatus 100 may perform transformation based on acoding unit or a transform unit included in the coding unit. The imageencoding apparatus 100 may transform residual data that is a differencevalue between an original value and a prediction value with respect topixels included in the coding unit. For example, the image encodingapparatus 100 may perform lossy-compression on the residual data throughquantization and discrete cosine transform (DCT)/discrete sine transform(DST). Alternatively, the image encoding apparatus 100 may performlossless-compression on the residual data without the quantization.

The image encoding apparatus 100 may determine a transform unit that isthe most efficient one for quantization and transformation. Thetransform unit in the coding unit may be recursively split into smallersized regions in a manner similar to that in which the coding unit issplit according to the tree structure, according to an embodiment. Thus,residual data in the coding unit may be split according to the transformunit having the tree structure according to transformation depths. Theimage encoding apparatus 100 may generate transformation splitinformation regarding splitting the coding unit and the transform unitaccording to the determined tree structure of the transform unit.

A transformation depth indicating the number of splitting times to reachthe transform unit by splitting the height and width of the coding unitmay also be set in the image encoding apparatus 100. For example, in acurrent coding unit of 2N×2N, a transformation depth may be 0 when thesize of a transform unit is 2N×2N, may be 1 when the size of thetransform unit is N×N, and may be 2 when the size of the transform unitis N/2×N/2. That is, the transform unit according to the tree structuremay be set according to the transformation depth.

In conclusion, the coding unit determiner 120 determines a predictionmethod that is the most efficient one for a current prediction unit andis from among a plurality of intra prediction methods and interprediction methods. Then, the coding unit determiner 120 determines aprediction unit determination scheme according to coding efficiencyaccording to a prediction result. Equally, the coding unit determiner120 determines a transform unit determination scheme according to codingefficiency according to a transformation result. According to the mostefficient prediction unit and transform unit determination scheme,coding efficiency of a coding unit is finally determined. The codingunit determiner 120 finalizes a hierarchical structure of a largestcoding unit, according to coding efficiency of a coding unit accordingto each depth.

The coding unit determiner 120 may measure coding efficiency of codingunits according to depths, prediction efficiency of prediction methods,or the like by using Rate-Distortion Optimization based on Lagrangianmultipliers.

The coding unit determiner 120 may generate split information indicatingwhether to split a coding unit according to each depth according to thedetermined hierarchical structure of the largest coding unit. Then, thecoding unit determiner 120 may generate, for split coding units,partition mode information to be used in determining a prediction unitand transform unit split information to be used in determining atransform unit. In addition, when the coding unit may be split by usingat least two split methods, the coding unit determiner 120 may generateboth split information and split shape information that indicates asplit method. The coding unit determiner 120 may generate informationregarding the prediction method and the transformation method that areused in the prediction unit and the transform unit.

The output unit 130 may output, in a bitstream, a plurality of pieces ofinformation generated by the largest coding unit determiner 110 and thecoding unit determiner 120 according to the hierarchical structure ofthe largest coding unit.

A method of determining the coding unit, the prediction unit, and thetransform unit according to the tree structure of the largest codingunit will be described below with reference to FIGS. 3 to 12.

FIG. 1B illustrates a block diagram of an image decoding apparatus 150based on coding units according to a tree structure, according to anembodiment.

The image decoding apparatus 150 includes a receiver 160, an encodinginformation extractor 170, and an image data decoder 180.

Definitions of the terms including a coding unit, a depth, a predictionunit, a transform unit, various split information, or the like for adecoding operation performed by the image decoding apparatus 150 areequal to those described above with reference to FIG. 1A and the imageencoding apparatus 100. Because the image decoding apparatus 150 isdesigned to reconstruct image data, various encoding methods used by theimage encoding apparatus 100 may also be applied to the image decodingapparatus 150.

The receiver 160 receives and parses a bitstream regarding an encodedvideo. The encoding information extractor 170 extracts, from the parsedbitstream, a plurality of pieces of information to be used in decodinglargest coding units, and provides them to the image data decoder 180.The encoding information extractor 170 may extract information regardinga largest size of a coding unit of a current picture from a header, asequence parameter set, or a picture parameter set of the currentpicture.

The encoding information extractor 170 extracts, from the parsedbitstream, a final depth and split information regarding coding unitsaccording to a tree structure according to each largest coding unit. Theextracted final depth and split information are output to the image datadecoder 180. The image data decoder 180 may split a largest coding unitaccording to the extracted final depth and split information, therebydetermining a tree structure of the largest coding unit.

The split information extracted by the encoding information extractor170 is split information regarding the tree structure determined togenerate a minimum encoding error, the determination being performed bythe image encoding apparatus 100. Therefore, the image decodingapparatus 150 may reconstruct an image by decoding data according to adecoding scheme that generates the minimum encoding error.

The encoding information extractor 170 may extract split informationregarding a data unit such as a prediction unit and a transform unitincluded in the coding unit. For example, the encoding informationextractor 170 may extract partition mode information regarding apartition mode that is the most efficient one for the prediction unit.The encoding information extractor 170 may extract transformation splitinformation regarding a tree structure that is the most efficient onefor the transform unit.

The encoding information extractor 170 may obtain information regardingthe most efficient prediction method with respect to prediction unitssplit from the coding unit. Then, the encoding information extractor 170may obtain information regarding the most efficient transformationmethod with respect to transform units split from the coding unit.

The encoding information extractor 170 extracts the information from thebitstream, according to a method of configuring the bitstream, themethod being performed by the output unit 130 of the image encodingapparatus 100.

The image data decoder 180 may split a largest coding unit into codingunits having the most efficient tree structure, based on the splitinformation. Then, the image data decoder 180 may split the coding unitinto the prediction units according to the partition mode information.The image data decoder 180 may split the coding unit into the transformunits according to the transformation split information.

The image data decoder 180 may predict the prediction units according tothe information regarding the prediction method. The image data decoder180 may perform inverse quantization and inverse transformation onresidual data that is a difference between an original value and aprediction value of a pixel, according to information regarding a methodof transforming a transform unit. The image data decoder 180 mayreconstruct pixels of the coding unit, according to a result of theprediction on the prediction units and a result of the transformation onthe transform units.

FIG. 2 illustrates a process of determining at least one coding unitwhen the image decoding apparatus 150 splits a current coding unit,according to an embodiment.

According to the present embodiment, the image decoding apparatus 150may determine, by using block shape information, a shape of a codingunit, and may determine, by using split shape information, a shapeaccording to which the coding unit is to be split. That is, a method ofsplitting a coding unit, which is indicated by the split shapeinformation, may be determined based on which block shape is indicatedby the block shape information used by the image decoding apparatus 150.

According to the present embodiment, the image decoding apparatus 150may use the block shape information indicating that a current codingunit has a square shape. For example, the image decoding apparatus 150may determine whether to split a square coding unit or not, whether tosplit the square coding unit vertically, whether to split the squarecoding unit horizontally, or whether to split the square coding unitinto four coding units, according to the split shape information.Referring to FIG. 2, when block shape information of a current codingunit 200 indicates a square shape, the image data decoder 180 may notsplit a coding unit 210 a having the same size as the current codingunit 200 according to split shape information indicating no split, ormay determine coding units 210 b, 210 c, and 210 d split based on splitshape information indicating a predetermined split method.

Referring to FIG. 2, the image decoding apparatus 150 may determine thetwo coding units 210 b obtained by splitting the current coding unit 200in a vertical direction based on split shape information indicatingsplit in a vertical direction, according to an embodiment. The imagedecoding apparatus 150 may determine the two coding units 210 c obtainedby splitting the current coding unit 200 in a horizontal direction basedon split shape information indicating split in a horizontal direction.The image decoding apparatus 150 may determine the four coding units 210d obtained by splitting the current coding unit 200 in vertical andhorizontal directions based on split shape information indicating splitin vertical and horizontal directions. However, a split shape forsplitting a square coding unit may not be limitedly interpreted to aboveshapes, and may include various shapes indicatable by split shapeinformation. Predetermined split shapes for splitting a square codingunit will be described in detail below through various embodiments.

FIG. 3 illustrates a process of determining at least one coding unitwhen the image decoding apparatus 150 splits a coding unit havingnon-square shape, according to an embodiment.

According to the present embodiment, the image decoding apparatus 150may use block shape information indicating that a current coding unithas a non-square shape. The image decoding apparatus 150 may determinewhether or not to split the current coding unit having the non-squareshape, or whether to split the current coding unit having the non-squareshape by using a predetermined method. Referring to FIG. 3, when blockshape information of a current coding unit 300 or 350 indicates anon-square shape, the image decoding apparatus 150 may not split acoding unit 310 or 360 having the same size as the current coding unit300 or 350 according to split shape information indicating no split, ormay determine coding units 320 a, 320 b, 330 a, 330 b, 330 c, 370 a, 370b, 380 a, 380 b, and 380 c split according to split shape informationindicating a predetermined split method. A predetermined split method ofsplitting a non-square coding unit will be described in detail belowthrough various embodiments.

According to the present embodiment, the image decoding apparatus 150may determine, by using the split shape information, a shape of a codingunit is split, and in this case, the split shape information mayindicate the number of at least one coding unit generated when a codingunit is split. Referring to FIG. 3, when the split shape informationindicates that the current coding unit 300 or 350 is split into twocoding units, the image decoding apparatus 150 may determine the twocoding units 320 a and 320 b or 370 a and 370 b, which are respectivelyincluded in the current coding unit 300 or 350 by splitting the currentcoding unit 300 or 350 based on the split shape information.

According to the present embodiment, when the image decoding apparatus150 splits the current coding unit 300 or 350 having the non-squareshape based on the split shape information, the image decoding apparatus150 may split the current coding unit 300 or 350 having the non-squareshape in consideration of a location of a longer side. For example, theimage decoding apparatus 150 may determine a plurality of coding unitsby splitting the current coding unit 300 or 350 in a direction ofsplitting the longer sides of the current coding unit 300 or 350 inconsideration of the shape of the current coding unit 300 or 350.

According to the present embodiment, when split shape informationindicates that a coding unit is split into an odd number of blocks, theimage decoding apparatus 150 may determine an odd number of coding unitsincluded in the current coding unit 300 or 350. For example, when splitshape information indicates that the current coding unit 300 or 350 issplit into three coding units, the image decoding apparatus 150 maysplit the current coding unit 300 or 350 into the three coding units 330a, 330 b, and 330 c or 380 a, 380 b, and 380 c. According to the presentembodiment, the image decoding apparatus 150 may determine the oddnumber of coding units included in the current coding unit 300 or 350,wherein sizes of the determined coding units are not the same. Forexample, a size of the coding unit 330 b or 380 b from among the oddnumber of coding units 330 a, 330 b, and 330 c or 380 a, 380 b, and 380c may be different from sizes of the coding units 330 a and 330 c or 380a or 380 c. That is, coding units that may be determined when thecurrent coding unit 300 or 350 is split may have different types ofsizes.

According to the present embodiment, when split shape informationindicates that a coding unit is split into an odd number of blocks, theimage decoding apparatus 150 may determine an odd number of coding unitsincluded in the current coding unit 300 or 350 and in addition, set apredetermined limit on at least one coding unit from among the oddnumber of coding units generated by splitting the current coding unit300 or 350. Referring to FIG. 3, the image decoding apparatus 150 maydecode the coding unit 330 b or 380 b at the center of the three codingunits 330 a, 330 b, and 330 c or 380 a, 380 b, and 380 c generated whenthe current coding unit 300 or 350 is split in a different manner fromthe coding units 330 a and 330 c or 380 a and 380 c. For example, theimage decoding apparatus 150 may limit the coding unit 330 b or 380 b atthe center not to be further split unlike the coding units 330 a and 330c or 380 a and 380 c, or to be split only a certain number of times.

FIG. 4 illustrates a process of splitting, by the image decodingapparatus 150, a coding unit based on at least one of block shapeinformation and split shape information, according to an embodiment.

According to the present embodiment, the image decoding apparatus 150may determine whether to split a first coding unit 400 having a squareshape into coding units based on at least one of block shape informationand split shape information. According to the present embodiment, whenthe split shape information indicates splitting of the first coding unit400 in a horizontal direction, the image decoding apparatus 150 maydetermine a second coding unit 410 by splitting the first coding unit400 in the horizontal direction. The terms “first coding unit”, “secondcoding unit”, and “third coding unit” according to an embodiment areused in the context of splitting a coding unit. For example, a secondcoding unit may be determined when a first coding unit is split and athird coding unit may be determined when the second coding unit issplit. Relationships between the first through third coding units usedhereinafter may be understood to follow the above order characteristics.

According to the present embodiment, the image decoding apparatus 150may determine whether to split the determined second coding unit 410into coding units based on at least one of block shape information andsplit shape information. Referring to FIG. 4, the image decodingapparatus 150 may split the second coding unit 410, which has anon-square shape determined by splitting the first coding unit 400, intoat least one third coding unit, for example, third coding units 420 a,420 b, 420 c, and 420 d, based on at least one of block shapeinformation and split shape information, or may not split the secondcoding unit 410. The image decoding apparatus 150 may obtain at leastone of block shape information and split shape information, the imagedecoding apparatus 150 may split the first coding unit 400 based on atleast one of the block shape information and the split shape informationto obtain a plurality of second coding units (for example, the secondcoding unit 410) having various shapes, and the second coding unit 410may be split according to a manner of splitting the first coding unit400 based on at least one of the block shape information and the splitshape information. According to the present embodiment, when the firstcoding unit 400 is split into the second coding units 410 based on atleast one of block shape information and split shape information aboutthe first coding unit 400, the second coding unit 410 may also be splitinto the third coding units, for example, the third coding units 420 a,420 b, and 420 c, 420 d, based on at least one of block shapeinformation and split shape information about the second coding unit410. That is, a coding unit may be recursively split based on at leastone of split shape information and block shape information related tothe coding unit. A method used to recursively split a coding unit willbe described below through various embodiments.

According to the present embodiment, the image decoding apparatus 150may determine to split each of the third coding units (for example, thethird coding units 420 a, 420 b, 420 c, and 420 d) into coding units ornot to split the second coding unit 410 based on at least one of blockshape information and split shape information. The image decodingapparatus 150 may split the second coding unit 410 having a non-squareshape into the odd number of third coding units 420 b, 420 c, and 420 d.The image decoding apparatus 150 may set a predetermined limitation on apredetermined third coding unit from among the odd number of thirdcoding units 420 b, 420 c, and 420 d. For example, the image decodingapparatus 150 may limit the coding unit 420 c located at the center fromamong the odd number of third coding units 420 b, 420 c, and 420 d to besplit no more or to be split to a settable number of times. Referring toFIG. 4, the image decoding apparatus 150 may limit the coding unit 420 clocated at the center from among the odd number of third coding units420 b, 420 c, and 420 d included in the second coding unit 410 having anon-square shape to be split no more, to be split into a predeterminedsplit manner (for example, split only into four coding units or splitinto a shape corresponding to that into which the second coding unit 410is split), or to be split only a predetermined number of times (forexample, split only n times, wherein n>0). However, the limitations onthe coding unit 420 c located at the center are simply embodiments, andthus the present disclosure should not be interpreted limitedly to theabove embodiments, and it should be interpreted that the limitationsinclude various limitations of decoding the coding unit 420 c located atthe center differently from the coding units 420 b and 420 d.

According to the present embodiment, the image decoding apparatus 150may obtain, from a predetermined location in a current coding unit, atleast one of block shape information and split shape information used tosplit the current coding unit.

FIG. 5 illustrates a method of determining, by the image decodingapparatus 150, a coding unit at a predetermined location from among anodd number of coding units, according to an embodiment.

According to the present embodiment, the image decoding apparatus 150may use information indicating a location of each of an odd number ofcoding units so as to determine a coding unit located at the center ofthe odd number of coding units. Referring to FIG. 5, the image decodingapparatus 150 may determine an odd number of coding units 520 a, 520 b,and 520 c by splitting a current coding unit 500. The image decodingapparatus 150 may determine the coding unit 520 b at the center by usinginformation about locations of the odd number of coding units 520 a, 520b, and 520 c. For example, the image decoding apparatus 150 maydetermine the coding unit 520 located at the center by determininglocations of the coding units 520 a, 520 b, and 520 c based oninformation indicating locations of predetermined samples included inthe coding units 520 a, 520 b, and 520 c. In detail, the image decodingapparatus 150 may determine the coding unit 520 b located at the centerby determining the locations of the coding units 520 a, 520 b, and 520 cbased on information indicating locations of upper left samples 530 a,530 b, and 530 c of the coding units 520 a, 520 b, and 520 c.

According to the present embodiment, the information indicating thelocations of the upper left samples 530 a, 530 b, and 530 c respectivelyincluded in the coding units 520 a, 520 b, and 520 c may includeinformation about locations or coordinates in a picture of the codingunits 520 a, 520 b, and 520 c. According to the present embodiment, theinformation indicating the locations of the upper left samples 530 a,530 b, and 530 c respectively included in the coding units 520 a, 520 b,and 520 c may include information indicating widths or heights of thecoding units 520 a, 520 b, and 520 c included in the current coding unit500, wherein the widths or heights may correspond to informationindicating differences between coordinates in the picture of the codingunits 520 a, 520 b, and 520 c. That is, the image decoding apparatus 150may determine the coding unit 520 b located at the center by directlyusing the information about the locations or coordinates in the pictureof the coding units 520 a, 520 b, and 520 c, or by using the informationabout the widths or heights of the coding units, which indicatedifference values between coordinates.

According to the present embodiment, the information indicating thelocation of the upper left sample 530 a of the top coding unit 520 a mayindicate (xa, ya) coordinates, information indicating the location ofthe upper left sample 530 b of the center coding unit 520 b may indicate(xb, yb) coordinates, and the information indicating the location of theupper left sample 530 c of the bottom coding unit 520 c may indicate(xc, yc) coordinates. The image decoding apparatus 150 may determine thecenter coding unit 520 b by using the coordinates of the upper leftsamples 530 a, 530 b, and 530 c respectively included in the codingunits 520 a, 520 b, and 520 c. For example, when the coordinates of theupper left samples 530 a, 530 b, and 530 c are aligned in an ascendingorder or descending order, the center coding unit 520 b including (xb,yb) that is coordinates of the upper left sample 530 b may be determinedas a coding unit located at the center from among the coding units 520a, 520 b, and 520 c determined when the current coding unit 500 issplit. Here, the coordinates indicating the locations of the upper leftsamples 530 a, 530 b, and 530 c may indicate coordinates indicatingabsolute locations in the picture, and further, may use (dxb, dyb)coordinates that are information indicating a relative location of theupper left sample 530 b of the center coding unit 520 b and (dxc, dyc)coordinates that are information indicating a relative location of theupper left sample 530 c of the bottom coding unit 520 c, based on thelocation of the upper left sample 530 c of the top coding unit 520 a.Also, a method of determining a coding unit at a predetermined locationby using coordinates of a sample included in a coding unit asinformation indicating a location of the sample should not be limitedlyinterpreted to the above method, and may be interpreted to variousarithmetic methods capable of using coordinates of a sample.

According to the present embodiment, the image decoding apparatus 150may split the current coding unit 500 into the plurality of coding units520 a, 520 b, and 520 c, and select a coding unit from among the codingunits 520 a, 520 b, and 520 c according to a predetermined criterion.For example, the image decoding apparatus 150 may select the coding unit520 b that has a different size from among the coding units 520 a, 520b, and 520 c.

According to the present embodiment, the image decoding apparatus 150may determine the width or height of each of the coding units 520 a, 520b, and 520 c by using the (xa, ya) coordinates that are the informationindicating the location of the upper left sample 530 a of the top codingunit 520 a, the (xb, yb) coordinates that are the information indicatingthe location of the upper left sample 530 b of the center coding unit520 b, and the (xc, yc) coordinates that are the information indicatingthe location of the upper left sample 530 c of the bottom coding unit520 c. The image decoding apparatus 150 may determine a size of each ofthe coding units 520 a, 520 b, and 520 c by using the coordinates (xa,ya), (xb, yb), and (xc, yc) indicating the locations of the coding units520 a, 520 b, and 520 c.

According to an embodiment, the image decoding apparatus 150 maydetermine the width of the top coding unit 520 a to xb-xa and the heightto yb-ya. According to the embodiment, the image decoding apparatus 150may determine the width of the center coding unit 520 b to xc-xb and theheight to yc-yb. According to the present embodiment, the image decodingapparatus 150 may determine the width or height of the bottom codingunit by using the width or height of the current coding unit, and thewidth and height of the top coding unit 520 a and the center coding unit520 b. The image decoding apparatus 150 may determine one coding unithaving a size different from other coding units based on the determinedwidths and heights of the coding units 520 a, 520 b, and 520 c.Referring to FIG. 5, the image decoding apparatus 150 may determine, asthe coding unit at the predetermined location, the center coding unit520 b having a size different from sizes of the top coding unit 520 aand the bottom coding unit 520 c. However, since a process ofdetermining, by the image decoding apparatus 150, a coding unit having asize different from other coding units is only an embodiment ofdetermining a coding unit at a predetermined location by using sizes ofcoding units determined based on sample coordinates, various processesof determining a coding unit at a predetermined location by comparingsizes of coding units determined according to predetermined samplecoordinates may be used.

However, a location of a sample considered to determine a location of acoding unit should not be limitedly interpreted to the upper left, butmay be interpreted that information about a location of an arbitrarysample included in a coding unit is usable.

According to the present embodiment, the image decoding apparatus 150may select a coding unit at a predetermined location from among an oddnumber of coding units that are determined when a current coding unit issplit, in consideration of a shape of the current coding unit. Forexample, when the current coding unit has a non-square shape in which awidth is longer than a height, the image decoding apparatus 150 maydetermine the coding unit at the predetermined location along ahorizontal direction. In other words, the image decoding apparatus 150may determine a coding unit from among coding units having differentlocations in the horizontal direction, and may set a limitation on thecoding unit. When the current coding unit has the non-square shape inwhich the height is longer than the width, the image decoding apparatus150 may determine the coding unit at the predetermined location along avertical direction. In other words, the image decoding apparatus 150 maydetermine a coding unit from among coding units having differentlocations in the vertical direction, and set a limitation on the codingunit.

According to the present embodiment, the image decoding apparatus 150may use information indicating a location of each of an even number ofcoding units so as to determine a coding unit at a predeterminedlocation from among the even number of coding units. The image decodingapparatus 150 may determine the even number of coding units by splittinga current coding unit, and determine the coding unit at thepredetermined location by using the information about the locations ofthe even number of coding units. Detailed processes thereof maycorrespond to processes of determining a coding unit at a predeterminedlocation (for example, a center location) from among an odd number ofcoding units, which have been described above with reference to FIG. 5,and thus descriptions thereof are not provided again.

According to the present embodiment, when a current coding unit having anon-square shape is split into a plurality of coding units,predetermined information about a coding unit at a predeterminedlocation may be used during a split process so as to determine thecoding unit at the predetermined location from among the plurality ofcoding units. For example, the image decoding apparatus 150 may use atleast one of block shape information and split shape information, whichare stored in a sample included in a center coding unit during a splitprocess so as to determine a coding unit located at the center fromamong a plurality of coding units obtained by splitting a current codingunit.

Referring to FIG. 5, the image decoding apparatus 150 may split thecurrent coding unit 500 into the plurality of coding units 520 a, 520 b,and 520 c based on at least one of block shape information and splitshape information, and determine the coding unit 520 b located at thecenter from among the plurality of coding units 520 a, 520 b, and 520 c.In addition, the image decoding apparatus 150 may determine the codingunit 520 b located at the center in consideration of a location where atleast one of the block shape information and the split shape informationis obtained. That is, at least one of the block shape information andthe split shape information of the current coding unit 500 may beobtained from the sample 540 located at the center of the current codingunit 500, and when the current coding unit 500 is split into theplurality of coding units 520 a, 520 b, and 520 c based on at least oneof the block shape information and the split shape information, thecoding unit 520 b including the sample 540 may be determined as thecoding unit located at the center. However, information used todetermine a coding unit located at the center should not be limitedlyinterpreted to at least one of block shape information and split shapeinformation, and various types of information may be used during aprocess of determining a coding unit located at the center.

According to the present embodiment, predetermined information foridentifying a coding unit at a predetermined location may be obtainedfrom a predetermined sample included in a coding unit to be determined.Referring to FIG. 5, the image decoding apparatus 150 may use at leastone of block shape information and split shape information obtained froma sample located at a predetermined location in the current coding unit500 (for example, a sample located at the center of the current codingunit 500) so as to determine a coding unit at a predetermined locationfrom among the plurality of coding units 520 a, 520 b, and 520 cdetermined when the current coding unit 500 is split (for example, acoding unit located at the center from among the plurality of codingunits). That is, the image decoding apparatus 150 may determine thesample at the predetermined location in consideration of a block shapeof the current coding unit 500, and the image decoding apparatus 150 maydetermine and set a predetermined limitation on the coding unit 520 bincluding the sample from which predetermined location (for example, atleast one of the block shape information and the split shapeinformation) is obtained, from among the plurality of coding units 520a, 520 b, and 520 c determined when the current coding unit 500 issplit. Referring to FIG. 5, the image decoding apparatus 150 maydetermine the sample 540 located at the center of the current codingunit 500, as the sample from which the predetermined information isobtained, and the image decoding apparatus 150 may set the predeterminedlocation during a decoding process, on the coding unit 520 b includingthe sample 540. However, a location of a sample from which predeterminedinformation is obtained should not be limitedly interpreted to the abovelocation, and the sample may be interpreted to samples at arbitrarylocations included in the coding unit 520 determined to be limited.

According to the present embodiment, a location of a sample from whichpredetermined location is obtained may be determined based on a shape ofthe current coding unit 500. According to the present embodiment, blockshape information may be used to determine whether a shape of a currentcoding unit is a square or a non-square, and a location of a sample fromwhich predetermined information is obtained may be determined based onthe shape. For example, the image decoding apparatus 150 may determine,as a sample from which predetermined information is obtained, a samplelocated on a boundary of splitting at least one of a width and a heightof a current coding unit into halves by using at least one ofinformation about the width of the current coding unit and informationabout the height of the current coding unit. As another example, whenblock shape information about a current coding unit indicates anon-square shape, the image decoding apparatus 150 may determine, as asample from which predetermined information is obtained, one of samplesadjacent to a boundary of splitting a longer side of the current codingunit into halves.

According to the present embodiment, when a current coding unit is splitinto a plurality of coding units, the image decoding apparatus 150 mayuse at least one of block shape information and split shape informationso as to determine a coding unit at a predetermined location from amongthe plurality of coding units. According to an embodiment, the imagedecoding apparatus 150 may obtain at least one of the block shapeinformation and the split shape information from a sample at apredetermined location included in the coding unit, and the imagedecoding apparatus 150 may split the plurality of coding units generatedwhen the current coding unit is split by using at least one of the splitshape information and the block shape information obtained from thesample at the predetermined location included in each of the pluralityof coding units. In other words, the coding unit may be recursivelysplit by using at least one of the block shape information and the splitshape information obtained from the sample at the predetermined locationin each coding unit. Since a recursive split process of a coding unithas been described above with reference to FIG. 4, details thereof arenot provided again.

According to the present embodiment, the image decoding apparatus 150may determine at least one coding unit by splitting a current codingunit, and determine an order of decoding the at least one coding unitaccording to a predetermined block (for example, a current coding unit).

FIG. 6 illustrates an order of processing a plurality of coding unitswhen the image decoding apparatus 150 determines the plurality of codingunits by splitting a current coding unit, according to an embodiment.

According to the present embodiment, the image decoding apparatus 150may determine, according to block shape information and split shapeinformation, second coding units 610 a and 610 b by splitting a firstcoding unit 600 in a vertical direction, second coding units 630 a and630 b by splitting the first coding unit 600 in a horizontal direction,or second coding units 650 a, 650 b, 650 c, and 650 d by splitting thefirst coding unit 600 in vertical and horizontal directions.

Referring to FIG. 6, the image decoding apparatus 150 may determine anorder such that the second coding units 610 a and 610 b determined bysplitting the first coding unit 600 in the vertical direction to beprocessed in a horizontal direction 610 c. The image decoding apparatus150 may determine a processing order of the second coding units 630 aand 630 b determined by splitting the first coding unit 600 in thehorizontal direction to be in a vertical direction 630 c. The imagedecoding apparatus 150 may determine the second coding units 650 a, 650b, 650 c, and 650 d determined by splitting the first coding unit 600 inthe vertical and horizontal directions to be processed according to apredetermined order (for example, a raster scan order or a z-scan order650 e) in which coding units in one row are processed and then codingunits in a next row are processed.

According to the present embodiment, the image decoding apparatus 150may recursively split coding units. Referring to FIG. 6, the imagedecoding apparatus 150 may determine a plurality of coding units 610 a,610 b, 630 a, 630 b, 650 a, 650 b, 650 c, and 650 d by splitting thefirst coding unit 600, and may recursively split each of the determinedplurality of coding units 610 a, 610 b, 630 a, 630 b, 650 a, 650 b, 650c, and 650 d. A method of splitting the plurality of coding units 610 a,610 b, 630 a, 630 b, 650 a, 650 b, 650 c, and 650 d may be similar to amethod of splitting the first coding unit 600. Accordingly, theplurality of coding units 610 a, 610 b, 630 a, 630 b, 650 a, 650 b, 650c, and 650 d may each be independently split into a plurality of codingunits. Referring to FIG. 6, the image decoding apparatus 150 maydetermine the second coding units 610 a and 610 b by splitting the firstcoding unit 600 in the vertical direction, and in addition, maydetermine to split or not to split each of the second coding units 610 aand 610 b independently.

According to the present embodiment, the image decoding apparatus 150may split the left second coding unit 610 a in the horizontal directionto obtain third coding units 620 a and 620 b, and may not split theright second coding unit 610 b.

According to the present embodiment, a processing order of coding unitsmay be determined based on a split process of coding units. In otherwords, a processing order of split coding units may be determined basedon a processing order of coding units just before being split. The imagedecoding apparatus 150 may determine an order of processing the thirdcoding units 620 a and 620 b determined when the left second coding unit610 a is split independently from the right second coding unit 610 b.Since the third coding units 620 a and 620 b are determined when theleft second coding unit 610 a is split in the horizontal direction, thethird coding units 620 a and 620 b may be processed in a verticaldirection 620 c. Also, since the order of processing the left secondcoding unit 610 a and the right second coding unit 610 b is in thehorizontal direction 610 c, the third coding units 620 a and 620 bincluded in the left second coding unit 610 a may be processed in thevertical direction 620 c and then the right second coding unit 610 b maybe processed. Because the above descriptions are for describing aprocess of determining a processing order according to coding unitsbefore being split, the process should not be limitedly interpreted tothe above embodiments, and various methods of independently processingcoding units split and determined in various shapes according to apredetermined order may be used.

FIG. 7 illustrates a process of determining, by the image decodingapparatus 150, that a current coding unit is split into an odd number ofcoding units when coding units are unable to be processed in apredetermined order, according to an embodiment.

According to the present embodiment, the image decoding apparatus 150may determine that the current coding unit is split into the odd numberof coding units based on obtained block shape information and splitshape information. Referring to FIG. 7, a first coding unit 700 having asquare shape may be split into second coding units 710 a and 710 bhaving non-square shapes, and the second coding units 710 a and 710 bmay be independently split into third coding units 720 a, 720 b, 720 c,720 d, and 720 e. According to the present embodiment, the imagedecoding apparatus 150 may determine a plurality of the third codingunits 720 a and 720 b by splitting the left coding unit 710 a from amongthe second coding units in a horizontal direction, and the right codingunit 710 b may be split into an odd number of the third coding units 720c, 720 d, and 720 e.

According to the present embodiment, the image decoding apparatus 150may determine whether a coding unit split into an odd number exists bydetermining whether the third coding units 720 a, 720 b, 720 c, 720 d,and 720 e are processable in a predetermined order. Referring to FIG. 7,the image decoding apparatus 150 may determine the third coding units720 a, 720 b, 720 c, 720 d, and 720 e by recursively splitting the firstcoding unit 700. The image decoding apparatus 150 may determine, basedon at least one of block shape information and split shape information,whether there is a coding unit split into an odd number from among thefirst coding unit 700, the second coding units 710 a and 710 b, and thethird coding units 720 a, 720 b, 720 c, 720 d, and 720 e. For example, acoding unit located at the right from among the second coding units 710a and 710 b may be split into the odd number of third coding units 720c, 720 d, and 720 e. An order of processing a plurality of coding unitsincluded in the first coding unit 700 may be a predetermined order 730(for example, a z-scan order), and the image decoding apparatus 150 maydetermine whether the third coding units 720 c, 720 d, and 720 edetermined when the right second coding unit 710 b is split into an oddnumber satisfy a condition of being processable according to thepredetermined order.

According to the present embodiment, the image decoding apparatus 150may determine whether the third coding units 720 a, 720 b, 720 c, 720 d,and 720 e included in the first coding unit 700 satisfy a condition ofbeing processable according to a predetermined order, wherein thecondition is related to whether at least one of a width and a height ofthe second coding units 710 a and 710 b is split into halves alongboundaries of the third coding units 720 a, 720 b, 720 c, 720 d, and 720e. For example, the third coding units 720 a and 720 b that aredetermined when the left second coding unit 710 a having a non-squareshape is split into halves satisfy the condition, but the third codingunits 720 c, 720 d, and 720 e do not satisfy the condition since theboundaries of the third coding units 720 c, 720 d, and 720 e that aredetermined when the right second coding unit 710 b is split into threecoding units are unable to split a width or height of the right secondcoding unit 710 b into halves. Also, the image decoding apparatus 150may determine disconnection of a scan order when the condition isdissatisfied, and determine that the right second coding unit 710 b issplit into an odd number of coding units based on the determinationresult. According to the present embodiment, when a coding unit is splitinto an odd number of coding units, the image decoding apparatus 150 mayset a predetermined limitation on a coding unit at a predeterminedlocation from among the coding units, and because details about thelimitation or the predetermined location have been described abovethrough various embodiments, details thereof are not provided again.

FIG. 8 illustrates a process of determining, by the image decodingapparatus 150, at least one coding unit when a first coding unit 800 issplit, according to an embodiment. According to the present embodiment,the image decoding apparatus 150 may split the first coding unit 800based on at least one of block shape information and split shapeinformation obtained through the receiver 160. The first coding unit 800having a square shape may be split into four coding units having squareshapes or non-square shapes. For example, referring to FIG. 8, whenblock shape information indicates that the first coding unit 800 is asquare and split shape information indicates that the first coding unit800 is split into non-square coding units, the image decoding apparatus150 may split the first coding unit 800 into a plurality of non-squarecoding units. In detail, when the split shape information indicates thatthe first coding unit 800 is split into a horizontal or verticaldirection to determine an odd number of coding units, the image decodingapparatus 150 may split the first coding unit 800 having a square shapeinto, as the odd number of coding units, second coding units 810 a, 810b, and 810 c determined when the first coding unit 800 is split in thevertical direction, or second coding units 820 a, 820 b, and 820 cdetermined when the first coding unit 800 is split in the horizontaldirection.

According to the present embodiment, the image decoding apparatus 150may determine whether the second coding units 810 a, 810 b, and 810 cand 820 a, 820 b, and 820 c included in the first coding unit 800satisfy a condition of being processable according to a predeterminedorder, wherein the condition is related to whether at least one of thewidth and the height of the first coding unit 800 is split into halvesalong the boundaries of the second coding units 810 a, 810 b, and 810 cand 820 a, 820 b, and 820 c. Referring to FIG. 8, since the boundariesof the second coding units 810 a, 810 b, and 810 c determined when thefirst coding unit 800 having a square shape is split in the verticaldirection are unable to split the width of the first coding unit 800into halves, it may be determined that the first coding unit 800 doesnot satisfy the condition of being processable according to thepredetermined order. Also, since the boundaries of the second codingunits 820 a, 820 b, and 820 c determined when the first coding unit 800having a square shape is split in the horizontal direction are unable tosplit the width of the first coding unit 800 into halves, it may bedetermined that the first coding unit 800 does not satisfy the conditionof being processable according to the predetermined order. When thecondition is dissatisfied, the image decoding apparatus 150 determinesdisconnection of a scan order and may determine that the first codingunit 800 is split into an odd number of coding units based on thedetermination result. According to the present embodiment, when a codingunit is split into an odd number of coding units, the image decodingapparatus 150 may set a predetermined limitation on a coding unit at apredetermined location from among the coding units, and since detailsabout the limitation or the predetermined location have been describedabove through various embodiments, details thereof are not providedagain.

According to the present embodiment, the image decoding apparatus 150may determine coding units having various shapes by splitting a firstcoding unit.

Referring to FIG. 8, the image decoding apparatus 150 may split thefirst coding unit 800 having a square shape and a first coding unit 830or 850 having a non-square shape into coding units having variousshapes.

FIG. 9 illustrates that, when a second coding unit having a non-squareshape, which is determined when a first coding unit 900 is split,satisfies a predetermined condition, a shape of the second coding unitthat is splittable is limited by the image decoding apparatus 150,according to an embodiment.

According to the present embodiment, the image decoding apparatus 150may determine, based on at least one of block shape information andsplit shape information obtained through the receiver 160, to split thefirst coding unit 900 having a square shape into second coding units 910a, 910 b, 920 a, and 920 b having non-square shapes. The second codingunits 910 a, 910 b, 920 a, and 920 b may be independently split.Accordingly, the image decoding apparatus 150 may determine to split ornot to split the second coding units 910 a, 910 b, 920 a, and 920 bbased on at least one of block shape information and split shapeinformation related to each of the second coding units 910 a, 910 b, 920a, and 920 b. According to the present embodiment, the image decodingapparatus 150 may determine third coding units 912 a and 912 b bysplitting the left second coding unit 910 a having a non-square shapeand determined when the first coding unit 900 is split in a verticaldirection. However, when the left second coding unit 910 a is split in ahorizontal direction, the image decoding apparatus 150 may limit theright second coding unit 910 b not to be split in the horizontaldirection like a direction in which the left second coding unit 910 a issplit. When the right second coding unit 910 b is split in the samedirection and third coding units 914 a and 914 b are determined, thethird coding units 912 a, 912 b, 914 a, and 914 b may be determined whenthe left second coding unit 910 a and the right second coding unit 910 bare independently split in the horizontal direction. However, this isthe same result as the image decoding apparatus 150 splitting the firstcoding unit 900 into four second coding units 930 a, 930 b, 930 c, and930 d having square shapes based on at least one of block shapeinformation and split shape information, and thus may be inefficient interms of image decoding.

According to the present embodiment, the image decoding apparatus 150may determine third coding units 922 a, 922 b, 924 a, and 924 b bysplitting the second coding units 920 a or 920 b having a non-squareshape and determined when the first coding unit 900 is split in thehorizontal direction. However, when one of second coding units (forexample, the top second coding unit 920 a) is split in the verticaldirection, the image decoding apparatus 150 may limit the other secondcoding unit (for example, the bottom second coding unit 920 b) not to besplit in the vertical direction like a direction in which the top secondcoding unit 920 a is split based on the above reasons.

FIG. 10 illustrates a process of splitting, by the image decodingapparatus 150, a coding unit having a square shape when split shapeinformation does not indicate splitting of the coding unit into fourcoding units having square shapes, according to an embodiment.

According to the present embodiment, the image decoding apparatus 150may determine second coding units 1010 a, 1010 b, 1020 a, 1020 b, andthe like by splitting a first coding unit 1000 based on at least one ofblock shape information and split shape information. The split shapeinformation may include information about various shapes into which acoding unit is splittable, but sometimes, the information about variousshapes may not include information for splitting a coding unit into foursquare coding units. According to such split shape information, theimage decoding apparatus 150 is unable to split the first coding unit1000 having a square shape into four square second coding units 1030 a,1030 b, 1030 c, and 1030 d. Based on the split shape information, theimage decoding apparatus 150 may determine the second coding units 1010a, 1010 b, 1020 a, 1020 b, and the like having non-square shapes.

According to the present embodiment, the image decoding apparatus 150may independently split the second coding units 1010 a, 1010 b, 1020 a,1020 b, and the like having non-square shapes. Each of the second codingunits 1010 a, 1010 b, 1020 a, 1020 b, and the like may be split in apredetermined order through a recursive method that may correspond to amethod of splitting the first coding unit 1000 based on at least one ofblock shape information and split shape information.

For example, the image decoding apparatus 150 may determine third codingunits 1012 a and 1012 b having square shapes by splitting the leftsecond coding unit 1010 a in a horizontal direction and may determinethird coding units 1014 a and 1014 b having square shapes by splittingthe right second coding unit 1010 b in a horizontal direction. Inaddition, the image decoding apparatus 150 may determine third codingunits 1016 a, 1016 b, 1016 c, and 1016 d having square shapes bysplitting both the left second coding unit 1010 a and the right secondcoding unit 1010 b in the horizontal direction. In this case, codingunits may be determined in the same manner in which the first codingunit 1000 is split into the four square second coding units 1030 a, 1030b, 1030 c, and 1030 d.

As another example, the image decoding apparatus 150 may determine thirdcoding units 1022 a and 1022 b having square shapes by splitting the topsecond coding unit 1020 a in the vertical direction and determine thirdcoding units 1024 a and 1024 b having square shapes by splitting thebottom second coding unit 1020 b in the vertical direction. In addition,the image decoding apparatus 150 may determine third coding units 1022a, 1022 b, 1024 a, and 1024 b having square shapes by splitting both thetop second coding unit 1020 a and the bottom second coding unit 1020 bin the vertical direction. In this case, coding units may be determinedin the same manner in which the first coding unit 1000 is split into thefour square second coding units 1030 a, 1030 b, 1030 c, and 1030 d.

FIG. 11 illustrates that a processing order between a plurality ofcoding units may be changed according to a split process of a codingunit, according to an embodiment.

According to the present embodiment, the image decoding apparatus 150may split a first coding unit 1100, based on block shape information andsplit shape information. When the block shape information indicates asquare shape and the split shape information indicates that the firstcoding unit 1100 is split in at least one of a horizontal direction anda vertical direction, the image decoding apparatus 150 may split thefirst coding unit 1100 to determine second coding units (for example,second coding units 1110 a, 1110 b, 1120 a, 1120 b, 1130 a, 1130 b, 1130c, 1130 d, and the like). Referring to FIG. 11, the second coding units1110 a, 1110 b, 1120 a, and 1120 b having non-square shapes anddetermined when the first coding unit 1100 is split only in thehorizontal or vertical direction may each be independently split basedon block shape information and split shape information about each of thesecond coding units 1110 a, 1110 b, 1120 a, and 1120 b. For example, theimage decoding apparatus 150 may determine third coding units 1116 a,1116 b, 1116 c, and 1116 d by splitting the second coding units 1110 aand 1110 b in the horizontal direction, wherein the second coding units1110 a and 1110 b are generated when the first coding unit 1100 is splitin the vertical direction, and may determine third coding units 1126 a,1126 b, 1126 c, and 1126 d by splitting the second coding units 1120 aand 1120 b in the horizontal direction, wherein the second coding units1120 a and 1120 b are generated when the first coding unit 1100 is splitin the horizontal direction. Because split processes of the secondcoding units 1110 a, 1110 b, 1120 a, and 1120 b have been described withreference to FIG. 9, details thereof are not provided again.

According to the present embodiment, the image decoding apparatus 150may process coding units according to a predetermined order. Sincecharacteristics about processing of coding units according to apredetermined order have been described above with reference to FIG. 6,details thereof are not provided again. Referring to FIG. 11, the imagedecoding apparatus 150 may determine four square third coding units 1116a, 1116 b, 1116 c, and 1116 d or 1126 a, 1126 b, 1126 c, and 1126 d bysplitting the first coding unit 1100 having a square shape. According tothe present embodiment, the image decoding apparatus 150 may determine aprocessing order of the third coding units 1116 a, 1116 b, 1116 c, and1116 d or 1126 a, 1126 b, 1126 c, and 1126 d according to a shape of thefirst coding unit 1100 being split.

According to the present embodiment, the image decoding apparatus 150may determine the third coding units 1116 a, 1116 b, 1116 c, and 1116 dby splitting each of the second coding units 1110 a and 1110 b in thehorizontal direction, wherein the second coding units 1110 a and 1110 bare generated when the first coding unit 1100 is split in the verticaldirection, and the image decoding apparatus 150 may process the thirdcoding units 1116 a, 1116 b, 1116 c, and 1116 d according to an order1117 of first processing the third coding units 1116 a and 1116 bincluded in the left second coding unit 1110 a in the vertical directionand then processing the third coding units 1116 c and 1116 d included inthe right second coding unit 1110 b in the vertical direction.

According to the present embodiment, the image decoding apparatus 150may determine the second coding units 1126 a, 1126 b, 1126 c, and 1126 dby splitting each of the second coding units 1120 a and 1120 b in thevertical direction, wherein the second coding units 1120 a and 1120 bare generated when the first coding unit 1100 is split in the horizontaldirection, and the image decoding apparatus 150 may process the thirdcoding units 1126 a, 1126 b, 1126 c, and 1126 d according to an order offirst processing the third coding units 1126 a and 1126 b included inthe top second coding unit 1120 a in the horizontal direction and thenprocessing the third coding units 1126 c and 1126 d included in thebottom second coding unit 1120 b in the horizontal direction.

Referring to FIG. 11, the third coding units 1116 a, 1116 b, 1116 c,1116 d, 1126 a, 1126 b, 1126 c, and 1126 d having square shapes may bedetermined when each of the second coding units 1110 a, 1110 b, 1120 a,and 1120 b are split. The second coding units 1110 a and 1110 bdetermined when the first coding unit 1100 is split in the verticaldirection and the second coding units 1120 a and 1120 b determined whenthe first coding unit 1100 is split in the horizontal direction havedifferent shapes, but according to the third coding units 1116 a, 1116b, 1116 c, 1116 d, 1126 a, 1126 b, 1126 c, and 1126 d determinedthereafter, the first coding unit 1100 is split into coding units havingthe same shapes. Accordingly, even when coding units having the sameshapes are determined as a result by recursively splitting coding unitsthrough different processes based on at least one of block shapeinformation and split shape information, the image decoding apparatus150 may process the coding units having the same shapes in differentorders.

FIG. 12 illustrates a process of determining a depth of a coding unitwhen a shape and size of the coding unit changes, in a case where aplurality of coding units are determined when the coding unit isrecursively split, according to an embodiment.

According to the present embodiment, the image decoding apparatus 150may determine a depth of a coding unit according to a predeterminedcriterion. For example, the predetermined criterion may be a length of alonger side of the coding unit. When a length of a longer side of acoding unit before being split is 2n times a length of a longer side ofa current coding unit, wherein n>0, the image decoding apparatus 150 maydetermine that a depth of the current coding unit is higher than a depthof the coding unit before being split by n. Hereinafter, a coding unithaving a higher depth will be referred to as a coding unit of a lowerdepth.

Referring to FIG. 12, according to the present embodiment, the imagedecoding apparatus 150 may determine a second coding unit 1202 and athird coding unit 1204 of lower depths by splitting a first coding unit1200 having a square shape, based on block shape information indicatinga square shape (for example, block shape information may indicate ‘0:SQUARE’). When a size of the first coding unit 1200 having a squareshape is 2N×2N, the second coding unit 1202 determined by splitting awidth and a height of the first coding unit 1200 by ½ may have a size ofN×N. In addition, the third coding unit 1204 determined by splitting awidth and a height of the second coding unit 1202 by ½ may have a sizeof N/2×N/2. In this case, a width and a height of the third coding unit1204 correspond to ½ times those of the first coding unit 1200. When adepth of the first coding unit 1200 is D, a depth of the second codingunit 1202, which is ½ times the width and height of the first codingunit 1200, may be D+1, and a depth of the third coding unit 1204, whichis ½ times the width and height of the first coding unit 1200, may beD+2.

According to the present embodiment, the image decoding apparatus 150may determine a second coding unit 1212 or 1222 and a third coding unit1214 or 1224 of lower depths by splitting a first coding unit 1210 or1220 having a non-square shape, based on block shape informationindicating a non-square shape (for example, the block shape informationmay indicate ‘1: NS_VER’ indicating that a height is longer than a widthor indicate ‘2: NS_HOR’ indicating that a width is longer than aheight).

The image decoding apparatus 150 may determine second coding units (forexample, the second coding units 1202, 1212, 1222, and the like) bysplitting at least one of the width and the height of the first codingunit 1210 having a size of N×2N. In other words, the image decodingapparatus 150 may determine the second coding unit 1202 having a size ofN×N or the second coding unit 1222 having a size of N×N/2 by splittingthe first coding unit 1210 in a horizontal direction, or may determinethe second coding unit 1212 having a size of N/2×N by splitting thefirst coding unit 1210 in horizontal and vertical directions.

According to the present embodiment, the image decoding apparatus 150may determine the second coding units (for example, the second codingunits 1202, 1212, 1222, and the like) by splitting at least one of thewidth and the height of the first coding unit 1220 having a size of2N×N. That is, the image decoding apparatus 150 may determine the secondcoding unit 1202 having a size of N×N or the second coding unit 1212having a size of N/2×N by splitting the first coding unit 1220 in thevertical direction, or may determine the second coding unit 1222 havinga size of N×N/2 by splitting the first coding unit 1220 in thehorizontal and vertical directions.

According to the present embodiment, the image decoding apparatus 150may determine third coding units (for example, the third coding units1204, 1214, 1224, and the like) by splitting at least one of a width anda height of the second coding unit 1202 having a size of N×N. That is,the image decoding apparatus 150 may determine the third coding unit1204 having a size of N/2×N/2, the third coding unit 1214 having a sizeof N/2×N/2, or the third coding unit 1224 having a size of N/2×N/2 bysplitting the second coding unit 1202 in vertical and horizontaldirections.

According to the present embodiment, the image decoding apparatus 150may determine the third coding units (for example, the third codingunits 1204, 1214, 1224, and the like) by splitting at least one of awidth and a height of the second coding unit 1212 having a size ofN/2×N. That is, the image decoding apparatus 150 may determine the thirdcoding unit 1204 having a size of N/2×N/2 or the third coding unit 1224having a size of N/2×N/2 by splitting the second coding unit 1212 in ahorizontal direction, or determine the third coding unit 1214 having asize of N/2×N/2 by splitting the second coding unit 1212 in vertical andhorizontal directions.

According to the present embodiment, the image decoding apparatus 150may determine the third coding units (for example, the third codingunits 1204, 1214, 1224, and the like) by splitting at least one of awidth and a height of the second coding unit 1214 having a size ofN×N/2. That is, the image decoding apparatus 150 may determine the thirdcoding unit 1204 having a size of N/2×N/2 or the third coding unit 1214having a size of N/2×N/2 by splitting the second coding unit 1212 in avertical direction, or determine the third coding unit 1224 having asize of N/2×N/2 by splitting the second coding unit 1212 in vertical andhorizontal directions.

According to the present embodiment, the image decoding apparatus 150may split coding units having square shapes (for example, the firstcoding units 1200, 1202, and 1204) in a horizontal or verticaldirection. For example, the first coding unit 1200 having a size of2N×2N may be split in the vertical direction to determine the firstcoding unit 1210 having a size of N×2N or in the horizontal direction todetermine the first coding unit 1220 having a size of 2N×N/. Accordingto the present embodiment, when a depth is determined based on a lengthof a longest side of a coding unit, a depth of a coding unit determinedwhen the first coding unit 1200, 1202, or 1204 is split in thehorizontal or vertical direction may be the same as a depth of the firstcoding unit 1200, 1202, or 1204.

According to the present embodiment, the width and height of the thirdcoding unit 1214 or 1224 may be ½ times the first coding unit 1210 or1220. When the depth of the first coding unit 1210 or 1220 is D, thedepth of the second coding unit 1212 or 1214, which is ½ times the widthand height of the first coding unit 1210 or 1220, may be D+1, and thedepth of the third coding unit 1214 or 1224, which is ½ times the widthand height of the first coding unit 1210 or 1220, may be D+2.

FIG. 13 illustrates a depth determinable according to shapes and sizesof coding units, and a part index (PID) for distinguishing between thecoding units, according to an embodiment.

According to the present embodiment, the image decoding apparatus 150may determine second coding units having various shapes by splitting afirst coding unit 1300 having a square shape. Referring to FIG. 13, theimage decoding apparatus 150 may determine second coding units 1302 a,1302 b, 1304 a, 1304 b, 1306 a, 1306 b, 1306 c, and 1306 d by splittingthe first coding unit 1300 in at least one of a vertical direction and ahorizontal direction, according to split shape information. That is, theimage decoding apparatus 150 may determine the second coding units 1302a, 1302 b, 1304 a, 1304 b, 1306 a, 1306 b, 1306 c, and 1306 d based onsplit shape information about the first coding unit 1300.

According to the present embodiment, depths of the second coding units1302 a, 1302 b, 1304 a, 1304 b, 1306 a, 1306 b, 1306 c, and 1306 ddetermined according to the split shape information about the firstcoding unit 1300 having a square shape may be determined based onlengths of longer sides. For example, since lengths of longer sides ofthe second coding units 1302 a, 1302 b, 1304 a, and 1304 b havingnon-square shapes are the same as a length of one side of the firstcoding unit 1300 having a square shape, depths of the first coding unit1300 and the second coding units 1302 a, 1302 b, 1304 a, and 1304 bhaving non-square shapes may be D, i.e., the same. On the other hand,when the image decoding apparatus 150 splits the first coding unit 1300into the four second coding units 1306 a, 1306 b, 1306 c, and 1306 dhaving square shapes based on split shape information, since a length ofone side of each of the second coding units 1306 a, 1306 b, 1306 c, and1306 d having square shapes is ½ of a length of one side of the firstcoding unit 1300, depths of the second coding units 1306 a, 1306 b, 1306c, and 1306 d may be D+1, i.e., one depth lower than the depth D of thefirst coding unit 1300.

According to the present embodiment, the image decoding apparatus 150may split a first coding unit 1310 having a height longer than a widthinto a plurality of second coding units 1312 a, 1312 b, 1314 a, 1314 b,and 1314 c by splitting the first coding unit 1310 in a horizontaldirection according to split shape information. According to the presentembodiment, the image decoding apparatus 150 may split a first codingunit 1320 having a width longer than a height into a plurality of secondcoding units 1322 a and 1322 b, or 1324 a, 1324 b, and 1324 c bysplitting the first coding unit 1320 in a vertical direction accordingto split shape information.

According to the present embodiment, depths of the second coding units1312 a, 1312 b, 1314 a, 1314 b, 1316 a, 1316 b, 1316 c, and 1316 ddetermined according to the split shape information about the firstcoding unit 1310 or 1320 having a non-square shape may be determinedbased on lengths of longer sides. For example, since a length of oneside of each of the second coding units 1312 a and 1312 b having squareshapes is ½ of a length of one side of the first coding unit 1310 havinga non-square shape in which a height is longer than a width, the depthsof the second coding units 1302 a, 1302 b, 1304 a, and 1304 b havingsquare shapes are D+1, i.e., one depth lower than the depth D of thefirst coding unit 1310 having a non-square shape.

In addition, the image decoding apparatus 150 may split the first codingunit 1310 having a non-square shape into an odd number of the secondcoding units 1314 a, 1314 b, and 1314 c based on split shapeinformation. The odd number of second coding units 1314 a, 1314 b, and1314 c may include the second coding units 1314 a and 1314 c havingnon-square shapes and the second coding unit 1314 b having a squareshape. Here, since lengths of longer sides of the second coding units1314 a and 1314 c having non-square shapes and a length of one side ofthe second coding unit 1314 b having a square shape are ½ of a length ofone side of the first coding unit 1310, depths of the second codingunits 1314 a, 1314 b, and 1314 c may be D+1, i.e., one depth lower thanthe depth D of the first coding unit 1310. The image decoding apparatus150 may determine depths of coding units related to the first codingunit 1310 having a non-square shape in which a width is longer than aheight in the similar manner as depths of coding units related to thefirst coding unit 1310 are determined.

According to the present embodiment, while determining PIDs fordistinguishing between coding units, the image decoding apparatus 150may determine the PIDs based on size ratios between the coding unitswhen an odd number of the coding units do not have the same size.Referring to FIG. 13, the coding unit 1314 b located at the center ofthe odd number of coding units 1314 a, 1314 b, and 1314 c has the samewidth as the coding units 1314 a and 1314 c, but has a height twicehigher than heights of the coding units 1314 a and 1314 c. In this case,the coding unit 1314 b located at the center may include two of each ofthe coding units 1314 a and 1314 c. Accordingly, when a PID of thecoding unit 1314 b located at the center according to a scan order is 1,a PID of the coding unit 1314 c located in a next order may be increasedby 2, i.e., 3. That is, values of PIDs may be discontinuous. Accordingto the present embodiment, the image decoding apparatus 150 maydetermine whether an odd number of coding units have the same size basedon discontinuity of PIDs for distinguishing between the coding units.

According to the present embodiment, the image decoding apparatus 150may determine whether a plurality of coding units determined when acurrent coding unit is split have certain split shapes based on valuesof PIDs for distinguishing between the coding units. Referring to FIG.13, the image decoding apparatus 150 may determine an even number of thecoding units 1312 a and 1312 b or an odd number of the coding units 1314a, 1314 b, and 1314 c by splitting the first coding unit 1310 having arectangular shape in which a height is longer than a width. The imagedecoding apparatus 150 may use an ID indicating each coding unit so asto distinguish between a plurality of coding units. According to thepresent embodiment, the PID may be obtained from a sample at apredetermined location (for example, an upper left sample) of eachcoding unit.

According to the present embodiment, the image decoding apparatus 150may determine a coding unit at a predetermined location from amongcoding units determined via split, by using PIDs for distinguishingbetween the coding units. According to an embodiment, when split shapeinformation about the first coding unit 1310 having a rectangular shapein which a height is longer than a width indicates split into threecoding units, the image decoding apparatus 150 may split the firstcoding unit 1310 into the three coding units 1314 a, 1314 b, and 1314 c.The image decoding apparatus 150 may allocate a PID to each of the threecoding units 1314 a, 1314 b, and 1314 c. The image decoding apparatus150 may compare PIDs of coding units so as to determine a center codingunit from among an odd number of coding units. The image decodingapparatus 150 may determine the coding unit 1314 b having a PIDcorresponding to a center value from among PIDs as a coding unit locatedat the center from among coding units determined when the first codingunit 1310 is split, based on PIDs of the coding units. According to thepresent embodiment, the image decoding apparatus 150 may determine PIDsbased on size ratios between coding units when the coding units do nothave the same size, while determining the PIDs for distinguishingbetween the coding units. Referring to FIG. 13, the coding unit 1314 bgenerated when the first coding unit 1310 is split may have the samewidth as the coding units 1314 a and 1314 c, but may have a height twicehigher than heights of the coding units 1314 a and 1314 c. In this case,when the PID of the coding unit 1314 b located at the center is 1, thePID of the coding unit 1314 c located in a next order may be increasedby 2, i.e., 3. As such, when an increase range changes while PIDs areuniformly increasing, the image decoding apparatus 150 may determinethat a coding unit is split into a plurality of coding units including acoding unit having a different size from other coding units. Accordingto the present embodiment, when split shape information indicates splitinto an odd number of coding units, the image decoding apparatus 150 maysplit a current coding unit into an odd number of coding units in whicha coding unit at a predetermined location (for example, a center codingunit) has a different size from other coding units. In this case, theimage decoding apparatus 150 may determine the center coding unit havingthe different size by using PIDs of the coding units. However, since thePID, and a size or location of a coding unit at a predetermined locationare specified to describe the present embodiment, and thus the presentdisclosure is not limited thereto, and various PIDs, and variouslocations and sizes of a coding unit may be used.

According to the present embodiment, the image decoding apparatus 150may use a predetermined data unit from which a coding unit starts to berecursively split.

FIG. 14 illustrates that a plurality of coding units are determinedaccording to a plurality of predetermined data units included in apicture, according to an embodiment.

According to the present embodiment, a predetermined data unit may bedefined as a data unit from which a coding unit starts to be recursivelysplit by using at least one of block shape information and split shapeinformation. That is, the predetermined data unit may correspond to acoding unit of an uppermost depth used while determining a plurality ofcoding units splitting a current picture. Hereinafter, for convenienceof description, such a predetermined data unit is referred to as areference data unit.

According to the present embodiment, a reference data unit may indicatea predetermined size and shape. According to an embodiment, a referencecoding unit may include M×N samples. Here, M and N may be equal to eachother, and may be an integer expressed as a multiple of 2. That is, areference data unit may indicate a square shape or a non-square shape,and may later be split into an integer number of coding units.

According to the present embodiment, the image decoding apparatus 150may split a current picture into a plurality of reference data units.According to the present embodiment, the image decoding apparatus 150may split the plurality of reference data units obtained by splittingthe current picture by using split information about each of thereference data units. Split processes of such reference data units maycorrespond to split processes using a quad-tree structure.

According to the present embodiment, the image decoding apparatus 150may pre-determine a smallest size available for the reference data unitincluded in the current picture. Accordingly, the image decodingapparatus 150 may determine the reference data unit having various sizesthat are equal to or larger than the smallest size, and determine atleast one coding unit based on the determined reference data unit byusing block shape information and split shape information.

Referring to FIG. 14, the image decoding apparatus 150 may use areference coding unit 1400 having a square shape, or may use a referencecoding unit 1402 having a non-square shape. According to the presentembodiment, a shape and size of a reference coding unit may bedetermined according to various data units (for example, a sequence, apicture, a slice, a slice segment, and a largest coding unit) that mayinclude at least one reference coding unit.

According to the present embodiment, the receiver 160 of the imagedecoding apparatus 150 may obtain, from a bitstream, at least one ofinformation about a shape of a reference coding unit and informationabout a size of the reference coding unit, according to the various dataunits. Processes of determining at least one coding unit included in thereference coding unit 1400 having a square shape have been describedabove through processes of splitting the current coding unit 1000 ofFIG. 10, and processes of determining at least one coding unit includedin the reference coding unit 1402 having a non-square shape have beendescribed above through processes of splitting the current coding unit1100 or 1150 of FIG. 11, and thus descriptions thereof are not providedhere.

According to the present embodiment, to determine a size and shape of areference coding unit according to some data units pre-determined basedon a predetermined condition, the image decoding apparatus 150 may use aPID for checking the size and shape of the reference coding unit. Thatis, the receiver 160 may obtain, from a bitstream, only a PID forchecking a size and shape of a reference coding unit as a data unitsatisfying a predetermined condition (for example, a data unit having asize equal to or smaller than a slice) from among various data units(for example, a sequence, a picture, a slice, a slice segment, and alargest coding unit), according to slices, slice segments, and largestcoding units. The image decoding apparatus 150 may determine the sizeand shape of the reference data unit according to data units thatsatisfy the predetermined condition, by using the PID. When informationabout a shape of a reference coding unit and information about a size ofa reference coding unit are obtained from a bitstream and used accordingto data units having relatively small sizes, usage efficiency of thebitstream may not be sufficient, and thus instead of directly obtainingthe information about the shape of the reference coding unit and theinformation about the size of the reference coding unit, only a PID maybe obtained and used. In this case, at least one of the size and theshape of the reference coding unit corresponding to the PID indicatingthe size and shape of the reference coding unit may be pre-determined.That is, the image decoding apparatus 150 may select at least one of thepre-determined size and shape of the reference coding unit according tothe PID so as to determine at least one of the size and shape of thereference coding unit included in a data unit that is a criterion forobtaining the PID.

According to the present embodiment, the image decoding apparatus 150may use at least one reference coding unit included in one largestcoding unit. That is, a largest coding unit splitting an image mayinclude at least one reference coding unit, and a coding unit may bedetermined when each of the reference coding unit is recursively split.According to the present embodiment, at least one of a width and heightof the largest coding unit may be an integer times at least one of awidth and height of the reference coding unit. According to the presentembodiment, a size of a reference coding unit may be equal to a size ofa largest coding unit, which is split n times according to a quad-treestructure. That is, the image decoding apparatus 150 may determine areference coding unit by splitting a largest coding unit n timesaccording to a quad-tree structure, and split the reference coding unitbased on at least one of block shape information and split shapeinformation according to various embodiments.

FIG. 15 illustrates a processing block that is a criterion indetermining a determining order of a reference coding unit included in apicture 1500, according to an embodiment.

According to the present embodiment, the image decoding apparatus 150may determine at least one processing block splitting a picture. Aprocessing block is a data unit including at least one reference codingunit splitting an image, and the at least one reference coding unitincluded in the processing block may be determined in a certain order.That is, a determining order of the at least one reference coding unitdetermined in each processing block may correspond to one of variousorders for determining a reference coding unit, and may vary accordingto processing blocks. A determining order of a reference coding unitdetermined per processing block may be one of various orders, such as araster scan order, a Z-scan order, an N-scan order, an up-right diagonalscan order, a horizontal scan order, and a vertical scan order, butshould not be limitedly interpreted by the scan orders.

According to the present embodiment, the image decoding apparatus 150may determine a size of at least one processing block included in animage by obtaining information about a size of a processing block. Theimage decoding apparatus 150 may obtain, from a bitstream, theinformation about a size of a processing block to determine the size ofthe at least one processing block included in the image. The size of theprocessing block may be a predetermined size of a data unit indicated bythe information about a size of a processing block.

According to the present embodiment, the receiver 160 of the imagedecoding apparatus 150 may obtain, from the bitstream, the informationabout a size of a processing block according to certain data units. Forexample, the information about a size of a processing block may beobtained from the bitstream in data units of images, sequences,pictures, slices, and slice segments. That is, the receiver 160 mayobtain, from the bitstream, the information about a size of a processingblock according to such several data units, and the image decodingapparatus 150 may determine the size of at least one processing blocksplitting the picture by using the obtained information about a size ofa processing block, wherein the size of the processing block may be aninteger times a size of a reference coding unit.

According to the present embodiment, the image decoding apparatus 150may determine sizes of processing blocks 1502 and 1512 included in thepicture 1500. For example, the image decoding apparatus 150 maydetermine a size of a processing block based on information about a sizeof a processing block, the information obtained from a bitstream.Referring to FIG. 15, the image decoding apparatus 150 may determinehorizontal sizes of the processing blocks 1502 and 1512 to be four timesa horizontal size of a reference coding unit, and a vertical sizethereof to be four times a vertical size of the reference coding unit,according to an embodiment. The image decoding apparatus 150 maydetermine a determining order of at least one reference coding unit inat least one processing block.

According to the present embodiment, the image decoding apparatus 150may determine each of the processing blocks 1502 and 1512 included inthe picture 1500 based on a size of a processing block, and maydetermine a determining order of at least one reference coding unitincluded in each of the processing blocks 1502 and 1512. According tothe present embodiment, determining of a reference coding unit mayinclude determining of a size of the reference coding unit.

According to the present embodiment, the image decoding apparatus 150may obtain, from a bitstream, information about a determining order ofat least one reference coding unit included in at least one processingblock, and may determine the determining order of the at least onereference coding unit based on the obtained information. The informationabout a determining order may be defined as an order or direction ofdetermining reference coding units in a processing block. That is, anorder of determining reference coding units may be independentlydetermined per processing block.

According to the present embodiment, the image decoding apparatus 150may obtain, from a bitstream, information about a determining order of areference coding unit according to certain data units. For example, thereceiver 160 may obtain, from the bitstream, the information about adetermining order of a reference coding unit according to data units,such as images, sequences, pictures, slices, slice segments, andprocessing blocks. Since the information about a determining order of areference coding unit indicates a determining order of a referencecoding unit in a processing block, the information about a determiningorder may be obtained per certain data unit including an integer numberof processing blocks.

According to the present embodiment, the image decoding apparatus 150may determine at least one reference coding unit based on the determinedorder.

According to the present embodiment, the receiver 160 may obtain, fromthe bitstream, information about a determining order of a referencecoding unit, as information related to the processing blocks 1502 and1512, and the image decoding apparatus 150 may determine an order ofdetermining at least one reference coding unit included in theprocessing blocks 1502 and 1512 and determine at least one referencecoding unit included in the picture 1500 according to a determiningorder of a coding unit. Referring to FIG. 15, the image decodingapparatus 150 may determine determining orders 1504 and 1514 of at leastone reference coding unit respectively related to the processing blocks1502 and 1512. For example, when information about a determining orderof a reference coding unit is obtained per processing block, determiningorders of a reference coding unit related to the processing blocks 1502and 1512 may be different from each other. When the determining order1504 related to the processing block 1502 is a raster scan order,reference coding units included in the processing block 1502 may bedetermined according to the raster scan order. On the other hand, whenthe determining order 1514 related to the processing block 1512 is aninverse order of a changed raster scan order, reference coding unitsincluded in the processing block 1512 may be determined in the inverseorder of the changed raster scan order. With reference to FIGS. 1 to 15,the method of splitting an image into largest coding units, andsplitting each largest coding unit into coding units having ahierarchical tree structure are described above. With reference to FIGS.16 to 25, it will now be described how to encode or decode the encodingunits of the same depth according to which encoding order.

FIG. 16 illustrates a video decoding apparatus 1600 involvingdetermining an encoding order of blocks, according to an embodiment.

The video decoding apparatus 1600 includes an encoding order informationobtainer 1610, an encoding order determiner 1620, and a decoder 1630.Referring to FIG. 16, the encoding order information obtainer 1610, theencoding order determiner 1620, and the decoder 1630 are formed asseparate elements, but in another embodiment, the encoding orderinformation obtainer 1610, the encoding order determiner 1620, and thedecoder 1630 may be integrated to be implemented as one element.

Referring to FIG. 16, the encoding order information obtainer 1610, theencoding order determiner 1620, and the decoder 1630 are seen aselements located within one apparatus, but the encoding orderinformation obtainer 1610, the encoding order determiner 1620, and thedecoder 1630 are not required to be physically adjacent to each other.Thus, in another embodiment, the encoding order information obtainer1610, the encoding order determiner 1620, and the decoder 1630 may bedispersed.

The encoding order information obtainer 1610, the encoding orderdeterminer 1620, and the decoder 1630 may be implemented by oneprocessor. In another embodiment, the encoding order informationobtainer 1610, the encoding order determiner 1620, and the decoder 1630may be implemented by a plurality of processors.

Functions performed by the encoding order information obtainer 1610, theencoding order determiner 1620, and the decoder 1630 may be performed bythe image data decoder 180 of FIG. 1B.

The encoding order information obtainer 1610 obtains encoding orderinformation about an encoding order of neighboring blocks.

Encoding order information indicates information about an encoding orderof blocks. The encoding order information may indicate whether to encodeat least two blocks according to a default encoding order. When theencoding order of the blocks does not follow the default encoding order,according to the encoding order information, the encoding order of theblocks is changed based on the encoding order information. On the otherhand, when the encoding order of the blocks follows the default encodingorder, according to the encoding order information, the encoding orderof the blocks is changed based on the default encoding order.

A default encoding order indicates an encoding order that is applied toall blocks when there is no encoding order information. The defaultencoding order may be an encoding order described with reference to FIG.15 or may be a changed encoding order.

The encoding order information may include an encoding order flag havinga 1-bit size which indicates the encoding order of the two blocks. Forexample, when the encoding order flag indicates 0, the encoding order ofthe two blocks is determined according to the default encoding order. Onthe other hand, when the encoding order flag indicates 1, the encodingorder of the two blocks is determined to be an inverse order to thedefault encoding order.

It is determined whether an encoding order of two blocks from among atleast three blocks follows the default encoding order, according to oneencoding order flag. Thus, an encoding order of the at least threeblocks may be determined according to a plurality of encoding orderflags.

However, to decrease bits of encoding order information, the encodingorder information may include a 1-bit encoding order flag indicating anencoding order of at least three spatially-neighboring blocks. Forexample, when the encoding order information indicates 0, the encodingorder of the at least three blocks is determined according to thedefault encoding order. On the other hand, when the encoding orderinformation indicates 1, the encoding order of the at least three blocksis determined to be an inverse order to the default encoding order.

The encoding order information obtainer 1610 may obtain the encodingorder information from a bitstream. When the bitstream does not includethe encoding order information, an encoding order of blocks may bedetermined according to the default encoding order. When the encodingorder information obtainer 1610 internally determines the encodingorder, according to an environment around a current block, the encodingorder information obtainer 1610 does not obtain the encoding orderinformation from the bitstream.

The encoding order information obtainer 1610 may check encoding orderchange allowance information with respect to an upper data unit of thecurrent block. The encoding order change allowance information indicateswhether a change in an encoding order is allowable for blocks includedin the upper data unit of the current block. When the encoding orderchange allowance information indicates that the change in the encodingorder is not allowable, all blocks of the upper data unit are decodedaccording to a default encoding order. When the encoding order changeallowance information indicates that encoding order information withrespect to the current block has been encoded, the encoding orderinformation obtainer 1610 may obtain the encoding order information.

In addition, it is possible to determine, based on the encoding orderchange allowance information, a meaning of a value indicated by anobtained encoding order flag. According to the encoding order changeallowance information, it is possible to determine to apply the encodingorder flag to two blocks, at least three blocks, or a plurality ofblocks based on another predetermined scheme.

When the encoding order change allowance information may be included ina video parameter set, a sequence parameter set, a picture parameterset, a slice segment header, a header of a largest coding unit, or thelike. When at least two types of the encoding order information arepresent, two pieces of encoding order change allowance informationregarding at least two types of the encoding order information may beseparately stored in different headers.

The encoding order determiner 1620 determines an encoding order ofblocks, based on the encoding order information.

When the encoding order flag is applied to the two blocks, the encodingorder determiner 1620 may determine an encoding order of the two blocks.For example, when the encoding order flag indicates 0 with respect to afirst block and a second block that has an encoding order after thefirst block according to a default encoding order, an encoding order ofthe first block and the second block is determined according to thedefault encoding order. However, when the encoding order flag indicates1, an encoding order of the first block is swapped with an encodingorder of the second block, thus, the second block is determined to havean encoding order preceding that of the first block.

When the encoding order flag is applied to at least three blocks, theencoding order determiner 1620 may determine a direction of an encodingorder of the at least three blocks. For example, when the encoding orderof the at least three blocks according to the default encoding orderindicates a first block, a second block, and then a third block, whenthe encoding order flag indicates 0, the encoding order of the at leastthree blocks is determined to be the first block, the second block, andthen the third block as to the default encoding order. On the otherhand, when the encoding order flag indicates 1, the encoding order ofthe at least three blocks is determined to be the third block, thesecond block, and then the first block, according to an inversedirection to the default encoding order.

The encoding order determiner 1620 may internally determine the encodingorder information according to an environment around the current block.For example, an encoding order of the current block may be determined byreferring to an encoding order that was applied to reconstructedneighboring blocks. Also, the encoding order may be determined,according to characteristics of the current block or the reconstructedneighboring blocks.

The decoder 1630 decodes blocks according to a determined encodingorder. The methods described with reference to FIGS. 2 to 15 may beapplied as a decoding method by the decoder 1630.

FIG. 17 illustrates a video encoding apparatus 1700 involvingdetermining an encoding order of blocks, according to an embodiment.

The video encoding apparatus 1700 includes an encoding order determiner1710, an encoder 1720, and an output unit 1730. Referring to FIG. 17,the encoding order determiner 1710, the encoder 1720, and the outputunit 1730 are formed as separate elements, but in another embodiment,the encoding order determiner 1710, the encoder 1720, and the outputunit 1730 may be integrated to be implemented as one element.

Referring to FIG. 17, the encoding order determiner 1710, the encoder1720, and the output unit 1730 are seen as elements located within oneapparatus, but the encoding order determiner 1710, the encoder 1720, andthe output unit 1730 are not required to be physically adjacent to eachother. Thus, in another embodiment, the encoding order determiner 1710,the encoder 1720, and the output unit 1730 may be dispersed.

The encoding order determiner 1710, the encoder 1720, and the outputunit 1730 may be implemented by one processor. In another embodiment,the encoding order determiner 1710, the encoder 1720, and the outputunit 1730 may be implemented by a plurality of processors.

Functions performed by the encoding order determiner 1710 and theencoder 1720 may be performed by the coding unit determiner 120 of FIG.1A. In addition, functions performed by the output unit 1730 of FIG. 17may be performed by the output unit 130 of FIG. 1A.

The encoding order determiner 1710 determines whether to change anencoding order of neighboring blocks. Coding efficiency of blocks basedon a default encoding order is compared with coding efficiency of theblocks based on an order changed from the default encoding order, andthen whether to change the encoding order may be determined. When thecoding efficiency based on the default encoding order is better, anencoding order based on the default encoding order is not changed. Onthe other hand, when the coding efficiency based on the order changedfrom the default encoding order is better, the encoding order based onthe default encoding order is changed.

The encoding order determiner 1710 may compare coding efficiencies ofpredetermined encoding orders. For example, whether to determine anencoding order of a two-block unit according to the default encodingorder may be determined. As another example, whether to determine anencoding order of a three-block unit according to the default encodingorder may be determined.

The encoding order determiner 1710 may internally determine encodingorder information, based on an environment around current blocks,according to a predetermined criterion. When an encoding order isinternally determined based on the environment around the currentblocks, a process of determining encoding order candidates may beomitted. In another embodiment, the encoding order determiner 1710 maydetermine the encoding order by analyzing neighboring blocks of acurrent block and a texture of the current block. Because the encodingorder is determined according to a similarity of the textures, codingefficiencies of the encoding order candidates are not always calculated.Therefore, calculation efficiency in the encoding process may beincreased.

When encoding order change allowance information of the current blockindicates that an encoding order change with respect to the currentblock is allowed, the encoding order determiner 1710 may determinewhether to change the encoding order. On the other hand, the encodingorder change is not allowed based on the encoding order change allowanceinformation, the encoding order determiner 1710 may determine anencoding order of all blocks, according to the default encoding order.

The encoder 1720 encodes the blocks according to the encoding orderdetermined by the encoding order determiner 1710.

The output unit 1730 outputs a bitstream including encoding orderinformation indicating whether the encoding order of the blocks ischanged from the default encoding order. In addition, the output unit1730 may output an encoding method for the blocks and encodinginformation about an encoding result.

A block with reference to FIGS. 16 and 17 may indicate a largest codingunit or a coding unit included in the largest coding unit. Withreference to FIG. 16, blocks may be sequentially adjacent to each otheraccording to the default encoding order. With reference to FIG. 16, theblocks may be horizontally or vertically adjacent to each other.

FIG. 18 is a diagram for describing the necessity of changing anencoding order of blocks. A connection line 1850 and a connection line1860 connect samples having similar characteristics. Thus, during aprocess of decoding a block 1800, high coding efficiency may be achievedby using encoding information corresponding to samples located at theconnection line 1850 and the connection line 1860. The connection line1850 in an embodiment at the left of FIG. 18 passes through a leftboundary and an upper boundary of a block 1810. The connection line 1850in an embodiment at the right of FIG. 18 passes through the leftboundary and a right boundary of the block 1810.

When the block 1800 is reconstructed according to a raster scan order,only left, upper, upper left, and upper right blocks of the block 1800have been reconstructed. Thus, only encoding information of the left,upper, upper left, and upper right blocks of the block 1800 may be usedin decoding the block 1800.

In the embodiment at the left of FIG. 18, the connection line 1850passes through an upper left block 1820 of the block 1800. Because theupper left block 1820 has been already reconstructed, the block 1800 maybe reconstructed based on the upper left block 1820. Thus, it is notnecessary for the block 1810 to be reconstructed prior to the block1800.

However, in the embodiment at the right of FIG. 18, the connection line1860 does not pass through blocks that have been reconstructed fromamong neighboring blocks of the block 1800. Thus, unless the block 1810is priorly reconstructed, the block 1800 cannot be efficientlyreconstructed. Thus, it is necessary to change an encoding order of theblock 1800 and the block 1810.

Because the connection line 1860 passes through an upper right block1840 of the block 1810, the block 1810 may be efficiently reconstructedby using encoding information of the upper right block 1840. After theblock 1810 is reconstructed, the block 1800 may be reconstructed byusing encoding information of the block 1810. Thus, in the embodiment atthe right of FIG. 18, it may be more efficient to reconstruct the block1810 prior to the block 1800.

FIGS. 19A and 19B illustrate embodiments of a method of determining anencoding order of blocks. FIG. 19A illustrates the embodiment of usingan encoding order flag indicating whether to change an encoding order oftwo neighboring blocks. FIG. 19B illustrates the embodiment of using anencoding order flag indicating whether a direction of an encoding orderof at least three neighboring blocks is changed.

An encoding order flag 1912 of FIG. 19A indicates whether an encodingorder of a block 1902 and a block 1904 is changed. When the encodingorder flag 1912 indicates 0, the block 1902 has an encoding orderpreceding the block 1904. On the other hand, when the encoding orderflag 1912 indicates 1, the block 1904 has an encoding order precedingthe block 1902. Equally, an encoding order flag 1914 determines anencoding order of the block 1904 and a block 1906, and an encoding orderflag 1916 determines an encoding order of the block 1906 and a block1908.

According to another embodiment, when the encoding order flag 1912indicates 0, the encoding order of the block 1902 and the block 1904 isnot changed, and only the block 1902 is reconstructed. Then, theencoding order of the block 1904 and the block 1906 which are adjacentto the reconstructed block 1902 is determined according to the encodingorder flag 1914.

When the encoding order flag 1912 indicates 1, the encoding order of theblock 1902 and the block 1904 is changed, and the block 1904 and theblock 1902 are sequentially reconstructed. Because the block 1904 hasbeen already reconstructed, the encoding order flag 1914 is not obtainedbut the encoding order flag 1916 is obtained. According to the encodingorder flag 1916, an encoding order of the block 1906 and the block 1908which are adjacent to the block 1904 is determined.

The aforementioned embodiment may be applied to all blocks of a line,regardless of locations and types of the blocks. Thus, until a block1910 is reconstructed, determination of an encoding order andreconstruction are performed.

An encoding order flag 1940 of FIG. 19B indicates a direction of anencoding order to be applied to all blocks. When the encoding order flag1940 indicates 0, a block 1922 located at the left is encoded accordingto a raster scan order. For example, after the block 1922 isreconstructed, a block 1924 may be reconstructed, and after the block1924 is reconstructed, a block 1926 may be reconstructed.

On the other hand, when the encoding order flag 1940 indicates 1, ablock 1930 located at the right is encoded according to an inverseraster scan order.

FIG. 20 illustrates a method of comparing coding efficiencies so as todetermine whether to change an encoding order of blocks. FIG. 20illustrates two blocks 2000 and 2020 to be encoded according to a rasterscan order, sub-blocks 2010 that are included in the block 2000 and areadjacent to a right boundary of the block 2000, and sub-blocks 2030 thatare included in the block 2020 and are adjacent to a left boundary ofthe block 2020.

According to the raster scan order, the block 2000 is encoded prior tothe block 2020. Thus, the sub-blocks 2010 included in the block 2000 areencoded prior to the sub-blocks 2030 included in the block 2020.

When only the encoding order of the block 2000 and the block 2020 ischanged without changing an encoding order of sub-blocks, the sub-blocks2030 are encoded without encoding information about left and rightsub-blocks such that coding efficiency of the sub-blocks 2030 may bedecreased. On the other hand, the sub-blocks 2010 are encoded by usingall of encoding information about left and right sub-blocks, codingefficiency of the sub-blocks 2010 may be increased.

Thus, whether a change in the encoding order of the block 2000 and theblock 2020 is efficient may be determined by summing an increase in thecoding efficiency of the sub-blocks 2010 and a decrease in the codingefficiency of the sub-blocks 2030. When the increase in the codingefficiency of the sub-blocks 2010 is greater than the decrease in thecoding efficiency of the sub-blocks 2030, it is better to change theencoding order of the block 2000 and the block 2020. However, on theother hand, the decrease in the coding efficiency of the sub-blocks 2030is greater than the increase in the coding efficiency of the sub-blocks2010, it is better not to change the encoding order of the block 2000and the block 2020.

FIG. 21 illustrates a reference sample to be used when a block 2100 ispredicted according to an intra mode.

With reference to FIG. 21, a first embodiment 2120 illustrates referencepixels 2102, 2106, 2108, and 2110 used in intra prediction when blocksin an upper row and a left block are reconstructed. In the firstembodiment 2120, the reference pixels 2102 and 2106 of the reconstructedupper blocks and the reference pixels 2108 of the reconstructed leftblock may be used in the intra prediction. The reference pixels 2110 ofa lower left block may be used only when the lower left block isreconstructed. To use the reference pixels 2102, 2106, 2108, and 2110,prediction directions included in a first intra prediction directiongroup 2125 may be used in intra predicting the block 2100.

A second embodiment 2130 illustrates reference pixels 2102, 2104, 2112,and 2114 used in intra prediction when blocks in an upper row and aright block are reconstructed. In the second embodiment 2130, thereference pixels 2102 and 2104 of the reconstructed upper blocks and thereference pixels 2112 of the reconstructed right block may be used inthe intra prediction. The reference pixels 2114 of a lower right blockmay be used only when the lower right block is reconstructed. To use thereference pixels 2102, 2104, 2112, and 2114, prediction directionsincluded in a second intra prediction direction group 2135 may be usedin intra predicting the current block 2100.

A third embodiment 2140 illustrates reference pixels 2102, 2108, and2112 used in intra prediction when an upper block, a right block, and aleft block are reconstructed. In the third embodiment 2140, thereference pixels 2102 of the upper block, the reference pixels 2108 ofthe left block, and the reference pixels 2112 of the right block may beused in the intra prediction. Prediction directions included in a thirdintra prediction direction group 2145 may be used in intra predictingthe block 2100.

According to the first embodiment 2120 and the second embodiment 2130,when the reference pixels 2110 of the lower left block and the referencepixels 2114 of the lower right block cannot be used, accuracy ofprediction may deteriorate. For example, in an upper-right directionmode, prediction according to the first embodiment 2120 may not beaccurate. However, in the third embodiment 2140, the used referencepixels 2102, 2108, and 2112 are all adjacent to the block 2100, thus,accuracy of prediction may be relatively high, compared to otherembodiments.

A fourth embodiment 2150 illustrates reference pixels 2102, 2104, and2106 used in intra prediction when only blocks in an upper row arereconstructed. In the fourth embodiment 2150, only the reference pixels2102, 2104, and 2106 of the reconstructed upper blocks may be used inthe intra prediction. Prediction directions included in a fourth intraprediction direction group 2155 may be used in intra predicting theblock 2100.

Unlike the third embodiment 2140, in the fourth embodiment 2150, thereference pixel 2102 of the upper block is the only pixel that isadjacent to the block 2100. Since the reference pixels 2104 and 2106 arespatially distant from the block 2100, accuracy of prediction maydeteriorate, compared to the first, second, and third embodiments 2120,2130, and 2140. Therefore, the intra prediction used in the fourthembodiment 2150 may be a vertical mode or a directional prediction modein a direction adjacent to the vertical mode which uses the referencepixel 2102 of the upper block that is adjacent to the block 2100.

FIGS. 22A to 22E illustrate an intra prediction method to be performedon a current block when a right block of the current block isreconstructed by changing an encoding order of blocks.

FIG. 22A illustrates an intra prediction method in a planar mode. Theplanar mode indicates an intra prediction mode in which a current pixelis predicted by performing double-interpolation on four pixels, based ona location of the current pixel, wherein the four pixels are from amongpixels adjacent to a current block and are at a same row or same columnof the current pixel. When an encoding order is not changed, a rightblock and a lower block of the current block are not reconstructed,thus, a lower left pixel 2201 and an upper right pixel 2202 are usedinstead of a lower pixel and a right pixel so as to predict the currentpixel. However, when the encoding order is changed such that the rightblock of the current block is reconstructed, instead of the upper rightpixel 2202, a pixel that is from among a right pixel 2203 and is locatedat the same row of the current pixel may be used as a reference pixel inthe planar mode.

FIG. 22B illustrates an intra prediction method in a DC mode. In the DCmode, an average value of pixels adjacent to a current block becomes aprediction value for pixels included in the current block. When anencoding order is not changed, a right block and a lower block of thecurrent block are not reconstructed, thus, left pixels 2211 and upperpixels 2212 are used in prediction according to the DC mode. However,when the encoding order is changed such that the right block of thecurrent block is reconstructed, right pixels 2213 may be additionallyused in the DC mode. Thus, an average value of pixel values of the leftpixels 2211, the upper pixels 2212, and the right pixels 2213 may beused as a prediction value for pixels included in the current block.

FIG. 22C illustrates an intra prediction method in a horizontal mode. Inthe horizontal mode, a pixel located in a horizontal direction of acurrent pixel is used in prediction of a current block. When an encodingorder is not changed, a right block of the current block is notreconstructed, thus, only left pixels 2221 are used in predictionaccording to the horizontal mode. However, when the encoding order ischanged such that the right block of the current block is reconstructed,right pixels 2222 may be additionally used in the horizontal mode. Forexample, a prediction value of the current pixel may be determined to bean average value of pixels that are from among the left pixels 2221 andthe right pixels 2222 and are located in a horizontal direction of thecurrent pixel. Alternatively, the prediction value of the current pixelmay be determined by interpolating the pixels according to a location ofthe current pixel, wherein the pixels are from among the left pixels2221 and the right pixels 2222 and are located in the horizontaldirection of the current pixel.

FIG. 22D illustrates an intra prediction method in an upper-rightdirection mode. In the upper-right direction mode, a pixel that islocated in an upper-right direction of a current pixel is used inprediction of the current pixel. When an encoding order is not changed,a right block of a current block is not reconstructed, thus, upper rightpixels 2232, instead of right pixels 2233, are used in predictionaccording to the upper-right direction mode. However, when the encodingorder is changed such that the right block of the current block isreconstructed, the right pixels 2233 may be used in the upper-rightdirection mode. Because the right pixels 2233 are closer to the currentblock, compared to the upper right pixels 2232, the current block may befurther accurately predicted.

FIG. 22E illustrates an intra prediction method in a lower-leftdirection mode. In the lower-left direction mode, a pixel that islocated in a lower-left direction of a current pixel is used inprediction of the current pixel. When an encoding order is not changed,a right block of a current block is not reconstructed, thus, only leftpixels 2241 and lower left pixels 2242 are used in prediction accordingto the lower-left direction mode. However, when the encoding order ischanged such that the right block of the current block is reconstructed,right pixels 2243 may be used in the lower-left direction mode. Forexample, an average value of a pixel and another pixel may be determinedto be a prediction value of the current pixel, wherein the pixel islocated in a lower-left direction of the current pixel and is from amongthe left pixels 2241 and the lower left pixels 2242 and the other pixelis located in an opposite direction of the lower-left direction and isfrom among the right pixels 2243. Alternatively, the prediction value ofthe current pixel may be determined by interpolating the pixel and theother pixel, based on a location of the current pixel, wherein the pixelis located in a lower-left direction of the current pixel and is fromamong the left pixels 2241 and the lower left pixels 2242 and the otherpixel is located in an opposite direction of the lower-left directionand is from among the right pixels 2243.

In an intra mode that is not described with reference to FIGS. 22A to22E, a method of using right pixels of a current block may be used toincrease accuracy of intra prediction.

FIG. 23 illustrates reference blocks to be used when a block 2300 ispredicted according to an inter mode.

Only when a left block of the block 2300 of FIG. 23 is reconstructed,some of motion vectors with respect to blocks including reference pixels2302, 2304, 2306, 2308, and 2310 are determined to be motion vectorcandidates of a first candidate list. One motion vector may be selectedfrom among the motion vector candidates of the first candidate list, anda plurality of pieces of encoding information such as a referencepicture index, or the like which are required for inter prediction maybe obtained from a block including the selected motion vector.

When a right block of the block 2300 is only reconstructed, some ofblocks including reference pixels 2302, 2310, 2312, 2316, and 2318 maybe determined to be motion vector candidates of a second candidate list.Alternatively, some of motion vectors with respect to blocks includingreference pixels 2302, 2308, 2310, 2314, and 2318 may be determined tobe the motion vector candidates of the second candidate list. One motionvector may be selected from among the motion vector candidates of thesecond candidate list, and a plurality of pieces of encoding informationsuch as a reference picture index, or the like which are required forinter prediction may be obtained from a block including the selectedmotion vector.

When the left and right blocks of the block 2300 are all reconstructed,an efficient candidate list from among the second candidate list and thefirst candidate list may be selected. Afterward, a motion vector may bedetermined from the selected candidate list. In another embodiment, whenthe left and right blocks of the current block 2300 are allreconstructed, a third candidate list that is different from the firstcandidate list and the second candidate list may be generated. Forexample, motion vectors with respect to blocks including referencepixels 2302, 2304, 2310, 2312, and 2316 may be included, as motionvector candidates, in the third candidate list.

When the left and right blocks of the block 2300 are not reconstructed,encoding information cannot be obtained from the left and right blocks.Therefore, a fourth candidate list including, as motion vectorcandidates, motion vectors with respect to blocks located at an upperrow of the block 2300 may be used. For example, motion vectors withrespect to blocks including reference pixels 2302, 2308, 2310, and 2312may be included, as motion vector candidates, in the fourth candidatelist.

In the Z encoding order, the inter prediction according to the firstcandidate list may be used. However, when an encoding order of twohorizontally-neighboring blocks is changed, a right block may be firstinter-predicted according to the second candidate list or the fourthcandidate list. After the right block is reconstructed, a left block maybe reconstructed by being inter-predicted according to one of the secondcandidate list and the third candidate list.

FIG. 24 illustrates a video decoding method performed by the videodecoding apparatus 1600, according to an embodiment.

In operation 2410, encoding order information indicating whether anencoding order of neighboring blocks is changed is obtained.

In operation 2420, an encoding order of blocks is determined, based onthe encoding order information.

In operation 2430, the blocks are decoded according to the determinedencoding order.

According to the present embodiment, the encoding order information mayindicate an encoding order of a first block and a second block that areadjacent to each other. When the encoding order information indicatesthat there is no change in the encoding order of the first block and thesecond block, the first block is first decoded. Then, encoding orderinformation about the second block and a third block adjacent to thesecond block may be obtained.

On the other hand, when the encoding order information indicates thatthere is a change in the encoding order of the first block and thesecond block, the second block is first decoded. After the second blockis decoded, the first block is decoded. After the first block isdecoded, encoding order information about a third block and a fourthblock may be obtained.

According to the present embodiment, encoding order information mayindicate a direction of an encoding order. When the encoding orderinformation indicates that the direction of the encoding order is equalto a default encoding order, blocks corresponding to the encoding orderinformation are decoded in a same direction as the default encodingorder. On the other hand, when the encoding order information indicatesthat the direction of the encoding order is opposite to the defaultencoding order, the blocks corresponding to the encoding orderinformation are decoded in an opposite direction to the default encodingorder.

The encoding order information may be implemented as a 1-bit sizeencoding order flag. Thus, the encoding order information may indicatewhether or not the encoding order is equal to the default encodingorder.

According to the present embodiment, the video decoding method mayfurther include obtaining encoding order change allowance informationindicating whether a change in an encoding order is allowed for blocksincluded in an upper data unit including a current block. Thus, onlywhen the encoding order change allowance information indicates that achange in an encoding order is allowed for the current block, encodingorder information about the current block may be obtained.

FIG. 25 illustrates a video encoding method performed by the videoencoding apparatus 1700, according to an embodiment.

In operation 2510, whether an encoding order of neighboring blocks ischanged is determined.

In operation 2520, blocks are encoded according to whether the encodingorder is changed.

In operation 2530, a bitstream including encoding order informationindicating whether the encoding order is changed and encodinginformation of the blocks is output.

The blocks of FIGS. 24 and 25 may be adjacent to each other in ahorizontal direction or a vertical direction. Also, the blocks of FIGS.24 and 25 may each be a largest coding unit or a data unit included in alargest coding unit.

According to the video encoding technique based on coding units having atree structure which is described with reference to FIGS. 1 through 25,image data of a spatial domain is encoded in each of the coding unitshaving a tree structure, and decoding is performed on each largestcoding unit according to the video decoding technique based on codingunits having a tree structure so that the image data of the spatialdomain is reconstructed, and by doing so, a picture and a video that isa picture sequence may be reconstructed. The reconstructed video may bereproduced by a reproducing apparatus, may be stored in a storagemedium, or may be transmitted through a network.

The embodiments according to the present disclosure may be written ascomputer programs and may be implemented in a general-use digitalcomputer that executes the programs by using a computer-readablerecording medium.

While the best embodiments of the present disclosure have beendescribed, it will be understood by one of ordinary skill in the artthat various replacements, modifications, or changes with respect to thepresent disclosure may be made therein without departing from the spiritand scope as defined by the following claims. That is, the claims willbe construed as including the various replacements, modifications, orchanges with respect to the present disclosure. Therefore, thedescriptions provided in the specification and drawings should beconsidered in a descriptive sense only and not for purposes oflimitation.

1. A video decoding apparatus comprising: one or more processorsconfigured to: obtain encoding order change allowance information withrespect to an upper data unit comprising a first block and a secondblock, wherein the encoding order change allowance information indicateswhether a change in an encoding order of blocks comprised in the upperdata unit is allowed, and when the encoding order change allowanceinformation indicates that the change in the encoding order of theblocks is allowed, obtain encoding order information indicating whetherthe first block is decoded prior to the second block, the first blockand the second block being adjacent to each other; an encoding orderdeterminer configured to: when the encoding order change allowanceinformation indicates that the change in the encoding order of theblocks is allowed, determine the encoding order of the first block andthe second block, based on the encoding order information, and when theencoding order change allowance information indicates that the change inthe encoding order of the blocks is not allowed, determine the encodingorder of the first block and the second block, based on a defaultencoding order; and a decoder configured to decode the first block andthe second block, according to the determined encoding order, wherein,the encoding order change allowance information is obtained from a videoparameter set, a sequence parameter set, a picture parameter set, or aslice segment header.